[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790304178325258-13942-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 02:42:58 dist-test-slave-kznw krb5kdc[13949](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 02:42:58 dist-test-slave-kznw krb5kdc[13949](info): set up 2 sockets
Sep 25 02:42:58 dist-test-slave-kznw krb5kdc[13949](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260925 02:42:59.470494 13942 mini_kdc.cc:121] Time spent starting KDC: real 0.956s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790304178325258-13942-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790304178325258-13942-0/krb5kdc/test-user.keytab.
Sep 25 02:42:59 dist-test-slave-kznw krb5kdc[13949](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304179, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T02:42:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T02:42:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.157.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.157.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.157.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab.
Entry for principal kudu/127.13.157.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab.
I20260925 02:42:59.591066 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790304178325258-13942-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790304178325258-13942-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790304178325258-13942-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.157.190:39231
--webserver_interface=127.13.157.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab
--principal=kudu/127.13.157.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:44233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.157.190:39231
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:42:59.741062 13962 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:42:59.741366 13962 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:42:59.741410 13962 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:42:59.745662 13962 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:42:59.745734 13962 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:42:59.745754 13962 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:42:59.745769 13962 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:42:59.745790 13962 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:42:59.751451 13962 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:44233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790304178325258-13942-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790304178325258-13942-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.157.190:39231
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab
--principal=kudu/127.13.157.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.157.190:39231
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.157.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13962
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790304178325258-13942-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:42:59.752722 13962 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:42:59.753963 13962 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:42:59.804687 13967 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:42:59.804633 13970 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:42:59.804702 13968 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:42:59.804840 13962 server_base.cc:1034] running on GCE node
I20260925 02:42:59.805550 13962 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:42:59.806834 13962 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:42:59.808007 13962 hybrid_clock.cc:648] HybridClock initialized: now 1790304179807964 us; error 54 us; skew 500 ppm
Sep 25 02:42:59 dist-test-slave-kznw krb5kdc[13949](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304179, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:42:59.811024 13962 init.cc:377] Logged in from keytab as kudu/127.13.157.190@KRBTEST.COM (short username kudu)
I20260925 02:42:59.812235 13962 webserver.cc:460] Webserver started at http://127.13.157.190:45513/ using document root <none> and password file <none>
I20260925 02:42:59.812837 13962 fs_manager.cc:359] Metadata directory not provided
I20260925 02:42:59.812906 13962 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:42:59.813118 13962 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:42:59.815400 13962 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790304178325258-13942-0/minicluster-data/master-0/data/instance:
uuid: "f2ae91fef2754e6ca0ed76287a17ded5"
format_stamp: "Formatted at 2026-09-25 02:42:59 on dist-test-slave-kznw"
server_key: "6d0ae221f9538827dfef5be3ff08b2e6"
server_key_iv: "00f83566dffecd489314b8ceb0043824"
server_key_version: "encryptionkey@0"
I20260925 02:42:59.816035 13962 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790304178325258-13942-0/minicluster-data/master-0/wal/instance:
uuid: "f2ae91fef2754e6ca0ed76287a17ded5"
format_stamp: "Formatted at 2026-09-25 02:42:59 on dist-test-slave-kznw"
server_key: "6d0ae221f9538827dfef5be3ff08b2e6"
server_key_iv: "00f83566dffecd489314b8ceb0043824"
server_key_version: "encryptionkey@0"
I20260925 02:42:59.820536 13962 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260925 02:42:59.825145 13977 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:42:59.826463 13962 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260925 02:42:59.826592 13962 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790304178325258-13942-0/minicluster-data/master-0/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790304178325258-13942-0/minicluster-data/master-0/wal
uuid: "f2ae91fef2754e6ca0ed76287a17ded5"
format_stamp: "Formatted at 2026-09-25 02:42:59 on dist-test-slave-kznw"
server_key: "6d0ae221f9538827dfef5be3ff08b2e6"
server_key_iv: "00f83566dffecd489314b8ceb0043824"
server_key_version: "encryptionkey@0"
I20260925 02:42:59.826707 13962 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790304178325258-13942-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790304178325258-13942-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790304178325258-13942-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:42:59.841861 13962 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:42:59.845371 13962 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:42:59.845552 13962 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:42:59.861958 13962 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.190:39231
I20260925 02:42:59.862020 14029 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.190:39231 every 8 connection(s)
I20260925 02:42:59.863160 13962 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
I20260925 02:42:59.868414 14030 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:42:59.872076 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 13962
I20260925 02:42:59.872203 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790304178325258-13942-0/minicluster-data/master-0/wal/instance
I20260925 02:42:59.873482 13942 external_mini_cluster.cc:1428] Setting key 4720c80bd379a20df5c571c9d52298cc
I20260925 02:42:59.875898 14030 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2ae91fef2754e6ca0ed76287a17ded5: Bootstrap starting.
I20260925 02:42:59.879285 14030 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f2ae91fef2754e6ca0ed76287a17ded5: Neither blocks nor log segments found. Creating new log.
I20260925 02:42:59.880502 14030 log.cc:824] T 00000000000000000000000000000000 P f2ae91fef2754e6ca0ed76287a17ded5: Log is configured to *not* fsync() on all Append() calls
I20260925 02:42:59.882999 14030 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2ae91fef2754e6ca0ed76287a17ded5: No bootstrap required, opened a new log
Sep 25 02:42:59 dist-test-slave-kznw krb5kdc[13949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304179, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.157.190@KRBTEST.COM
I20260925 02:42:59.896121 14030 raft_consensus.cc:348] T 00000000000000000000000000000000 P f2ae91fef2754e6ca0ed76287a17ded5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2ae91fef2754e6ca0ed76287a17ded5" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 39231 } }
I20260925 02:42:59.896421 14030 raft_consensus.cc:374] T 00000000000000000000000000000000 P f2ae91fef2754e6ca0ed76287a17ded5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:42:59.896493 14030 raft_consensus.cc:729] T 00000000000000000000000000000000 P f2ae91fef2754e6ca0ed76287a17ded5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2ae91fef2754e6ca0ed76287a17ded5, State: Initialized, Role: FOLLOWER
I20260925 02:42:59.897032 14030 consensus_queue.cc:260] T 00000000000000000000000000000000 P f2ae91fef2754e6ca0ed76287a17ded5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2ae91fef2754e6ca0ed76287a17ded5" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 39231 } }
I20260925 02:42:59.897190 14030 raft_consensus.cc:388] T 00000000000000000000000000000000 P f2ae91fef2754e6ca0ed76287a17ded5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 02:42:59.897255 14030 raft_consensus.cc:482] T 00000000000000000000000000000000 P f2ae91fef2754e6ca0ed76287a17ded5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 02:42:59.897356 14030 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f2ae91fef2754e6ca0ed76287a17ded5 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:42:59.898424 14030 raft_consensus.cc:504] T 00000000000000000000000000000000 P f2ae91fef2754e6ca0ed76287a17ded5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2ae91fef2754e6ca0ed76287a17ded5" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 39231 } }
I20260925 02:42:59.898820 14030 leader_election.cc:302] T 00000000000000000000000000000000 P f2ae91fef2754e6ca0ed76287a17ded5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f2ae91fef2754e6ca0ed76287a17ded5; no voters:
I20260925 02:42:59.899230 14030 leader_election.cc:288] T 00000000000000000000000000000000 P f2ae91fef2754e6ca0ed76287a17ded5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 02:42:59.899703 14035 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f2ae91fef2754e6ca0ed76287a17ded5 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:42:59.899945 14035 raft_consensus.cc:686] T 00000000000000000000000000000000 P f2ae91fef2754e6ca0ed76287a17ded5 [term 1 LEADER]: Becoming Leader. State: Replica: f2ae91fef2754e6ca0ed76287a17ded5, State: Running, Role: LEADER
I20260925 02:42:59.900051 14032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:42:59.876978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53072 (local address 127.13.157.190:39231)
0925 02:42:59.877525 (+ 547us) server_negotiation.cc:206] Beginning negotiation
0925 02:42:59.877544 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0925 02:42:59.878723 (+ 1179us) server_negotiation.cc:407] Connection header received
0925 02:42:59.883395 (+ 4672us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:42:59.883413 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:42:59.883838 (+ 425us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:42:59.884111 (+ 273us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:42:59.887273 (+ 3162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:42:59.888355 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:42:59.889761 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:42:59.890276 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:42:59.890402 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:42:59.894544 (+ 4142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:42:59.894585 (+ 41us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:42:59.894600 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:42:59.894634 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:42:59.896369 (+ 1735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:42:59.896970 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:42:59.896975 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:42:59.896979 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:42:59.897092 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:42:59.897488 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:42:59.897490 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:42:59.897491 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:42:59.897896 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:42:59.898091 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0925 02:42:59.898679 (+ 588us) server_negotiation.cc:299] Negotiation successful
0925 02:42:59.898989 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":378,"thread_start_us":170,"threads_started":1}
I20260925 02:42:59.900312 14035 consensus_queue.cc:237] T 00000000000000000000000000000000 P f2ae91fef2754e6ca0ed76287a17ded5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2ae91fef2754e6ca0ed76287a17ded5" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 39231 } }
I20260925 02:42:59.901014 14030 sys_catalog.cc:564] T 00000000000000000000000000000000 P f2ae91fef2754e6ca0ed76287a17ded5 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:42:59.903874 14036 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2ae91fef2754e6ca0ed76287a17ded5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f2ae91fef2754e6ca0ed76287a17ded5. Latest consensus state: current_term: 1 leader_uuid: "f2ae91fef2754e6ca0ed76287a17ded5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2ae91fef2754e6ca0ed76287a17ded5" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 39231 } } }
I20260925 02:42:59.903949 14037 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2ae91fef2754e6ca0ed76287a17ded5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f2ae91fef2754e6ca0ed76287a17ded5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2ae91fef2754e6ca0ed76287a17ded5" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 39231 } } }
I20260925 02:42:59.904248 14037 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2ae91fef2754e6ca0ed76287a17ded5 [sys.catalog]: This master's current role is: LEADER
I20260925 02:42:59.904244 14036 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2ae91fef2754e6ca0ed76287a17ded5 [sys.catalog]: This master's current role is: LEADER
W20260925 02:42:59.908115 14049 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f2ae91fef2754e6ca0ed76287a17ded5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 02:42:59.908202 14049 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 02:42:59.908303 14050 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:42:59.909344 14050 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:42:59.915014 14050 catalog_manager.cc:1347] Generated new cluster ID: 5976243d9656404abfb198e70766a7b5
I20260925 02:42:59.915072 14050 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 02:42:59.936957 14050 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 02:42:59.937860 14050 catalog_manager.cc:1504] Loading token signing keys...
I20260925 02:42:59.947736 14050 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f2ae91fef2754e6ca0ed76287a17ded5: Generated new TSK 0
I20260925 02:42:59.948493 14050 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.157.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.157.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.157.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.129.keytab.
Entry for principal kudu/127.13.157.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.129.keytab.
I20260925 02:42:59.977862 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790304178325258-13942-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790304178325258-13942-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790304178325258-13942-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790304178325258-13942-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.157.129:0
--local_ip_for_outbound_sockets=127.13.157.129
--webserver_interface=127.13.157.129
--webserver_port=0
--tserver_master_addrs=127.13.157.190:39231
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.129.keytab
--principal=kudu/127.13.157.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:44233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:43:00.110953 14056 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:43:00.111420 14056 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:43:00.111478 14056 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:43:00.116402 14056 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:43:00.116472 14056 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:43:00.116536 14056 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.157.129
I20260925 02:43:00.122810 14056 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:44233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790304178325258-13942-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790304178325258-13942-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.129.keytab
--principal=kudu/127.13.157.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.157.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790304178325258-13942-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.157.129
--webserver_port=0
--tserver_master_addrs=127.13.157.190:39231
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14056
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.157.129
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790304178325258-13942-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:43:00.124056 14056 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:43:00.125216 14056 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:43:00.135507 14064 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:00.135522 14061 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:00.135547 14062 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:43:00.135767 14056 server_base.cc:1034] running on GCE node
I20260925 02:43:00.136243 14056 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:43:00.136844 14056 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:43:00.138000 14056 hybrid_clock.cc:648] HybridClock initialized: now 1790304180137949 us; error 66 us; skew 500 ppm
Sep 25 02:43:00 dist-test-slave-kznw krb5kdc[13949](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304180, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:43:00.141109 14056 init.cc:377] Logged in from keytab as kudu/127.13.157.129@KRBTEST.COM (short username kudu)
I20260925 02:43:00.142066 14056 webserver.cc:460] Webserver started at http://127.13.157.129:36805/ using document root <none> and password file <none>
I20260925 02:43:00.142637 14056 fs_manager.cc:359] Metadata directory not provided
I20260925 02:43:00.142702 14056 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:43:00.142932 14056 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:43:00.145267 14056 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790304178325258-13942-0/minicluster-data/ts-0/data/instance:
uuid: "ffc10c870993474c8153be245247507c"
format_stamp: "Formatted at 2026-09-25 02:43:00 on dist-test-slave-kznw"
server_key: "27a7927deb539b6b27b3e1a8b3c441f9"
server_key_iv: "68b3918647eb9b93fd09bec2c8f4ce59"
server_key_version: "encryptionkey@0"
I20260925 02:43:00.145869 14056 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790304178325258-13942-0/minicluster-data/ts-0/wal/instance:
uuid: "ffc10c870993474c8153be245247507c"
format_stamp: "Formatted at 2026-09-25 02:43:00 on dist-test-slave-kznw"
server_key: "27a7927deb539b6b27b3e1a8b3c441f9"
server_key_iv: "68b3918647eb9b93fd09bec2c8f4ce59"
server_key_version: "encryptionkey@0"
I20260925 02:43:00.150573 14056 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260925 02:43:00.153354 14071 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:00.154834 14056 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260925 02:43:00.154978 14056 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790304178325258-13942-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790304178325258-13942-0/minicluster-data/ts-0/wal
uuid: "ffc10c870993474c8153be245247507c"
format_stamp: "Formatted at 2026-09-25 02:43:00 on dist-test-slave-kznw"
server_key: "27a7927deb539b6b27b3e1a8b3c441f9"
server_key_iv: "68b3918647eb9b93fd09bec2c8f4ce59"
server_key_version: "encryptionkey@0"
I20260925 02:43:00.155089 14056 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790304178325258-13942-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790304178325258-13942-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790304178325258-13942-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:43:00.182974 14056 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:43:00.186199 14056 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:43:00.186360 14056 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:43:00.186980 14056 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:43:00.187865 14056 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:43:00.187947 14056 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:00.188014 14056 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:43:00.188045 14056 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:00.202129 14056 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.129:44061
I20260925 02:43:00.202200 14184 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.129:44061 every 8 connection(s)
I20260925 02:43:00.203235 14056 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790304178325258-13942-0/minicluster-data/ts-0/data/info.pb
I20260925 02:43:00.208472 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 14056
I20260925 02:43:00.208563 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790304178325258-13942-0/minicluster-data/ts-0/wal/instance
I20260925 02:43:00.208853 13942 external_mini_cluster.cc:1428] Setting key 0d8db857c179b1410d99cb8299ee6bd3
Sep 25 02:43:00 dist-test-slave-kznw krb5kdc[13949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304180, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.129@KRBTEST.COM for kudu/127.13.157.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.157.130@KRBTEST.COM; defaulting to no policy
I20260925 02:43:00.221781 14032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:00.205223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.129:58087 (local address 127.13.157.190:39231)
0925 02:43:00.205401 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:00.205405 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:00.206182 (+ 777us) server_negotiation.cc:407] Connection header received
0925 02:43:00.209245 (+ 3063us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:00.209249 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:00.209307 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:00.209375 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:00.210972 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:00.212721 (+ 1749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:00.214028 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:00.214272 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:00.214374 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:00.217736 (+ 3362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:00.217739 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:00.217741 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:43:00.217773 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:00.219424 (+ 1651us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:00.220104 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:00.220108 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:00.220110 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:00.220207 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:00.220553 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:00.220556 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:00.220557 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:00.220812 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:00.220898 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:00.221462 (+ 564us) server_negotiation.cc:299] Negotiation successful
0925 02:43:00.221597 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.157.130@KRBTEST.COM" created.
I20260925 02:43:00.222832 14187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:00.205576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:39231 (local address 127.13.157.129:58087)
0925 02:43:00.206041 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:00.206084 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:00.208940 (+ 2856us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:00.209585 (+ 645us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:00.209592 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:00.210106 (+ 514us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:00.210757 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:00.210775 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:00.212857 (+ 2082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:00.212860 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:00.213886 (+ 1026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:00.213899 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:00.214439 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:00.214444 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:00.214656 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:00.215543 (+ 887us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:00.215577 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:00.217526 (+ 1949us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:43:00.219577 (+ 2051us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:00.219584 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:00.219602 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:00.219992 (+ 390us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:00.220286 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:00.220288 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:00.220290 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:00.220439 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:00.220931 (+ 492us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:43:00.220937 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:43:00.221206 (+ 269us) client_negotiation.cc:769] Sending connection context
0925 02:43:00.221385 (+ 179us) client_negotiation.cc:240] Negotiation successful
0925 02:43:00.221761 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":108,"threads_started":1}
I20260925 02:43:00.224344 14185 heartbeater.cc:347] Connected to a master server at 127.13.157.190:39231
I20260925 02:43:00.224620 14185 heartbeater.cc:464] Registering TS with master...
I20260925 02:43:00.225281 14185 heartbeater.cc:511] Master 127.13.157.190:39231 requested a full tablet report, sending...
I20260925 02:43:00.227691 13994 ts_manager.cc:194] Registered new tserver with Master: ffc10c870993474c8153be245247507c (127.13.157.129:44061)
I20260925 02:43:00.229530 13994 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.157.129@KRBTEST.COM'} at 127.13.157.129:58087
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.157.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.130.keytab.
Entry for principal kudu/127.13.157.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.130.keytab.
I20260925 02:43:00.243515 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790304178325258-13942-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790304178325258-13942-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790304178325258-13942-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790304178325258-13942-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.157.130:0
--local_ip_for_outbound_sockets=127.13.157.130
--webserver_interface=127.13.157.130
--webserver_port=0
--tserver_master_addrs=127.13.157.190:39231
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.130.keytab
--principal=kudu/127.13.157.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:44233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:43:00.371301 14190 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:43:00.371598 14190 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:43:00.371656 14190 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:43:00.375715 14190 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:43:00.375780 14190 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:43:00.375865 14190 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.157.130
I20260925 02:43:00.381307 14190 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:44233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790304178325258-13942-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790304178325258-13942-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.130.keytab
--principal=kudu/127.13.157.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.157.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790304178325258-13942-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.157.130
--webserver_port=0
--tserver_master_addrs=127.13.157.190:39231
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14190
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.157.130
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790304178325258-13942-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:43:00.382393 14190 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:43:00.383493 14190 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:43:00.393445 14195 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:00.393414 14198 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:00.393452 14196 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:43:00.393682 14190 server_base.cc:1034] running on GCE node
I20260925 02:43:00.394138 14190 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:43:00.394776 14190 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:43:00.395934 14190 hybrid_clock.cc:648] HybridClock initialized: now 1790304180395913 us; error 33 us; skew 500 ppm
Sep 25 02:43:00 dist-test-slave-kznw krb5kdc[13949](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304180, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:43:00.398965 14190 init.cc:377] Logged in from keytab as kudu/127.13.157.130@KRBTEST.COM (short username kudu)
I20260925 02:43:00.399942 14190 webserver.cc:460] Webserver started at http://127.13.157.130:46495/ using document root <none> and password file <none>
I20260925 02:43:00.400527 14190 fs_manager.cc:359] Metadata directory not provided
I20260925 02:43:00.400616 14190 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:43:00.400836 14190 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:43:00.403288 14190 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790304178325258-13942-0/minicluster-data/ts-1/data/instance:
uuid: "ee854a6436dc40249c33248d220af8c9"
format_stamp: "Formatted at 2026-09-25 02:43:00 on dist-test-slave-kznw"
server_key: "4a5fa6780ce02b03873a25f2a55bae41"
server_key_iv: "2b7fde92e7aba5a1bb61794a741214af"
server_key_version: "encryptionkey@0"
I20260925 02:43:00.403909 14190 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790304178325258-13942-0/minicluster-data/ts-1/wal/instance:
uuid: "ee854a6436dc40249c33248d220af8c9"
format_stamp: "Formatted at 2026-09-25 02:43:00 on dist-test-slave-kznw"
server_key: "4a5fa6780ce02b03873a25f2a55bae41"
server_key_iv: "2b7fde92e7aba5a1bb61794a741214af"
server_key_version: "encryptionkey@0"
I20260925 02:43:00.408006 14190 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260925 02:43:00.410624 14205 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:00.412063 14190 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260925 02:43:00.412240 14190 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790304178325258-13942-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790304178325258-13942-0/minicluster-data/ts-1/wal
uuid: "ee854a6436dc40249c33248d220af8c9"
format_stamp: "Formatted at 2026-09-25 02:43:00 on dist-test-slave-kznw"
server_key: "4a5fa6780ce02b03873a25f2a55bae41"
server_key_iv: "2b7fde92e7aba5a1bb61794a741214af"
server_key_version: "encryptionkey@0"
I20260925 02:43:00.412351 14190 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790304178325258-13942-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790304178325258-13942-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790304178325258-13942-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:43:00.425662 14190 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:43:00.429179 14190 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:43:00.429404 14190 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:43:00.430066 14190 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:43:00.431090 14190 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:43:00.431145 14190 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:00.431200 14190 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:43:00.431231 14190 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:00.442154 14190 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.130:34361
I20260925 02:43:00.442266 14318 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.130:34361 every 8 connection(s)
I20260925 02:43:00.443146 14190 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790304178325258-13942-0/minicluster-data/ts-1/data/info.pb
I20260925 02:43:00.449946 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 14190
I20260925 02:43:00.450055 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790304178325258-13942-0/minicluster-data/ts-1/wal/instance
I20260925 02:43:00.450325 13942 external_mini_cluster.cc:1428] Setting key 60758c5226ca0129ad100fd88f71846b
Sep 25 02:43:00 dist-test-slave-kznw krb5kdc[13949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304180, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.130@KRBTEST.COM for kudu/127.13.157.190@KRBTEST.COM
I20260925 02:43:00.458127 14032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:00.445053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.130:38609 (local address 127.13.157.190:39231)
0925 02:43:00.445250 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:00.445254 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:00.446034 (+ 780us) server_negotiation.cc:407] Connection header received
0925 02:43:00.447402 (+ 1368us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:00.447406 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:00.447501 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:00.447574 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:00.448984 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:00.449553 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:00.451161 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:00.451513 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:00.451619 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:00.454443 (+ 2824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:00.454447 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:00.454450 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:43:00.454480 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:00.455989 (+ 1509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:00.456492 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:00.456496 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:00.456498 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:00.456594 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:00.456935 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:00.456938 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:00.456941 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:00.457187 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:00.457291 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:00.457797 (+ 506us) server_negotiation.cc:299] Negotiation successful
0925 02:43:00.457934 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":22,"server-negotiator.queue_time_us":59}
I20260925 02:43:00.458984 14321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:00.445438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:39231 (local address 127.13.157.130:38609)
0925 02:43:00.445885 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:00.445929 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:00.447191 (+ 1262us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:00.447773 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:00.447780 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:00.448226 (+ 446us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:00.448829 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:00.448846 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:00.449687 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:00.449691 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:00.451031 (+ 1340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:00.451042 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:00.451667 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:00.451671 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:00.451866 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:00.452473 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:00.452492 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:00.454252 (+ 1760us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:43:00.456119 (+ 1867us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:00.456125 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:00.456140 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:00.456394 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:00.456696 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:00.456698 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:00.456700 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:00.456850 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:00.457311 (+ 461us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:43:00.457316 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:43:00.457562 (+ 246us) client_negotiation.cc:769] Sending connection context
0925 02:43:00.457723 (+ 161us) client_negotiation.cc:240] Negotiation successful
0925 02:43:00.458010 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":105,"threads_started":1}
I20260925 02:43:00.460166 14319 heartbeater.cc:347] Connected to a master server at 127.13.157.190:39231
I20260925 02:43:00.460441 14319 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.13.157.131@KRBTEST.COM; defaulting to no policy
I20260925 02:43:00.461035 14319 heartbeater.cc:511] Master 127.13.157.190:39231 requested a full tablet report, sending...
I20260925 02:43:00.462131 13994 ts_manager.cc:194] Registered new tserver with Master: ee854a6436dc40249c33248d220af8c9 (127.13.157.130:34361)
I20260925 02:43:00.462704 13994 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.157.130@KRBTEST.COM'} at 127.13.157.130:38609
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.157.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.157.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.131.keytab.
Entry for principal kudu/127.13.157.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.131.keytab.
I20260925 02:43:00.478598 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790304178325258-13942-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790304178325258-13942-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790304178325258-13942-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790304178325258-13942-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.157.131:0
--local_ip_for_outbound_sockets=127.13.157.131
--webserver_interface=127.13.157.131
--webserver_port=0
--tserver_master_addrs=127.13.157.190:39231
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.131.keytab
--principal=kudu/127.13.157.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:44233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:43:00.605520 14324 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:43:00.605798 14324 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:43:00.605862 14324 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:43:00.609951 14324 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:43:00.610020 14324 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:43:00.610103 14324 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.157.131
I20260925 02:43:00.615816 14324 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:44233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790304178325258-13942-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790304178325258-13942-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.131.keytab
--principal=kudu/127.13.157.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.157.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790304178325258-13942-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.157.131
--webserver_port=0
--tserver_master_addrs=127.13.157.190:39231
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14324
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.157.131
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790304178325258-13942-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:43:00.617007 14324 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:43:00.618050 14324 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:43:00.628199 14330 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:00.628180 14332 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:43:00.628309 14324 server_base.cc:1034] running on GCE node
W20260925 02:43:00.628182 14329 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:43:00.628886 14324 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:43:00.629565 14324 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:43:00.630736 14324 hybrid_clock.cc:648] HybridClock initialized: now 1790304180630708 us; error 39 us; skew 500 ppm
Sep 25 02:43:00 dist-test-slave-kznw krb5kdc[13949](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304180, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:43:00.633440 14324 init.cc:377] Logged in from keytab as kudu/127.13.157.131@KRBTEST.COM (short username kudu)
I20260925 02:43:00.634271 14324 webserver.cc:460] Webserver started at http://127.13.157.131:37461/ using document root <none> and password file <none>
I20260925 02:43:00.634799 14324 fs_manager.cc:359] Metadata directory not provided
I20260925 02:43:00.634855 14324 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:43:00.635057 14324 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:43:00.637027 14324 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790304178325258-13942-0/minicluster-data/ts-2/data/instance:
uuid: "e4cefc9fc6b0417fa31bd86411df7820"
format_stamp: "Formatted at 2026-09-25 02:43:00 on dist-test-slave-kznw"
server_key: "2a121135de859c77fd6fb9a48decc5ae"
server_key_iv: "52d9ce204dd2988439a246ba1586aa36"
server_key_version: "encryptionkey@0"
I20260925 02:43:00.637548 14324 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790304178325258-13942-0/minicluster-data/ts-2/wal/instance:
uuid: "e4cefc9fc6b0417fa31bd86411df7820"
format_stamp: "Formatted at 2026-09-25 02:43:00 on dist-test-slave-kznw"
server_key: "2a121135de859c77fd6fb9a48decc5ae"
server_key_iv: "52d9ce204dd2988439a246ba1586aa36"
server_key_version: "encryptionkey@0"
I20260925 02:43:00.641227 14324 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260925 02:43:00.643879 14339 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:00.645153 14324 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 02:43:00.645283 14324 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790304178325258-13942-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790304178325258-13942-0/minicluster-data/ts-2/wal
uuid: "e4cefc9fc6b0417fa31bd86411df7820"
format_stamp: "Formatted at 2026-09-25 02:43:00 on dist-test-slave-kznw"
server_key: "2a121135de859c77fd6fb9a48decc5ae"
server_key_iv: "52d9ce204dd2988439a246ba1586aa36"
server_key_version: "encryptionkey@0"
I20260925 02:43:00.645383 14324 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790304178325258-13942-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790304178325258-13942-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790304178325258-13942-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:43:00.663228 14324 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:43:00.666465 14324 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:43:00.666626 14324 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:43:00.667426 14324 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:43:00.668393 14324 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:43:00.668449 14324 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:00.668491 14324 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:43:00.668505 14324 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:00.680003 14324 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.131:37607
I20260925 02:43:00.680138 14452 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.131:37607 every 8 connection(s)
I20260925 02:43:00.680927 14324 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790304178325258-13942-0/minicluster-data/ts-2/data/info.pb
I20260925 02:43:00.685389 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 14324
I20260925 02:43:00.685496 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790304178325258-13942-0/minicluster-data/ts-2/wal/instance
I20260925 02:43:00.685807 13942 external_mini_cluster.cc:1428] Setting key 00383b1ff4afb65dd745938ea7c6ef84
Sep 25 02:43:00 dist-test-slave-kznw krb5kdc[13949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304180, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.131@KRBTEST.COM for kudu/127.13.157.190@KRBTEST.COM
I20260925 02:43:00.695132 14032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:00.682720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.131:47309 (local address 127.13.157.190:39231)
0925 02:43:00.682891 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:00.682896 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:00.684101 (+ 1205us) server_negotiation.cc:407] Connection header received
0925 02:43:00.685583 (+ 1482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:00.685587 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:00.685651 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:00.685747 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:00.687373 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:00.688142 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:00.689308 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:00.689521 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:00.689603 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:00.691965 (+ 2362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:00.691968 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:00.691970 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:43:00.691995 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:00.693148 (+ 1153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:00.693689 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:00.693692 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:00.693693 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:00.693774 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:00.694133 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:00.694137 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:00.694138 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:00.694336 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:00.694422 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:00.694876 (+ 454us) server_negotiation.cc:299] Negotiation successful
0925 02:43:00.695006 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260925 02:43:00.695977 14455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:00.683272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:39231 (local address 127.13.157.131:47309)
0925 02:43:00.683914 (+ 642us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:00.683971 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:00.685333 (+ 1362us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:00.685910 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:00.685916 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:00.686392 (+ 476us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:00.687221 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:00.687239 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:00.688267 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:00.688270 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:00.689198 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:00.689208 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:00.689636 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:00.689639 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:00.689783 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:00.690313 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:00.690329 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:00.691813 (+ 1484us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:43:00.693266 (+ 1453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:00.693271 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:00.693283 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:00.693573 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:00.693862 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:00.693865 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:00.693866 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:00.694043 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:00.694445 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:43:00.694451 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:43:00.694684 (+ 233us) client_negotiation.cc:769] Sending connection context
0925 02:43:00.694835 (+ 151us) client_negotiation.cc:240] Negotiation successful
0925 02:43:00.695126 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":222,"threads_started":1}
I20260925 02:43:00.697137 14453 heartbeater.cc:347] Connected to a master server at 127.13.157.190:39231
I20260925 02:43:00.697382 14453 heartbeater.cc:464] Registering TS with master...
I20260925 02:43:00.697840 14453 heartbeater.cc:511] Master 127.13.157.190:39231 requested a full tablet report, sending...
I20260925 02:43:00.698792 13994 ts_manager.cc:194] Registered new tserver with Master: e4cefc9fc6b0417fa31bd86411df7820 (127.13.157.131:37607)
I20260925 02:43:00.699471 13994 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.157.131@KRBTEST.COM'} at 127.13.157.131:47309
I20260925 02:43:00.700143 13942 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 25 02:43:00 dist-test-slave-kznw krb5kdc[13949](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304180, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 25 02:43:00 dist-test-slave-kznw krb5kdc[13949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304180, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.13.157.129@KRBTEST.COM
I20260925 02:43:00.725607 14460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:00.716279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55702 (local address 127.13.157.129:44061)
0925 02:43:00.716531 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:00.716546 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:00.716595 (+ 49us) server_negotiation.cc:407] Connection header received
0925 02:43:00.716724 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:00.716731 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:00.716951 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:00.717080 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:00.717683 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:00.718460 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:00.719290 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:00.719709 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:00.719829 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:00.722137 (+ 2308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:00.722151 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:00.722168 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:43:00.722196 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:00.723869 (+ 1673us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:00.724282 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:00.724289 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:00.724291 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:00.724373 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:00.724618 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:00.724622 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:00.724623 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:00.724878 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:00.725095 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:00.725288 (+ 193us) server_negotiation.cc:299] Negotiation successful
0925 02:43:00.725442 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":59,"threads_started":1}
W20260925 02:43:00.726636 14099 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55702
Sep 25 02:43:00 dist-test-slave-kznw krb5kdc[13949](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304180, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 25 02:43:00 dist-test-slave-kznw krb5kdc[13949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304180, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.157.129@KRBTEST.COM
I20260925 02:43:00.752959 14460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:00.744721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55712 (local address 127.13.157.129:44061)
0925 02:43:00.744895 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:00.744901 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:00.744984 (+ 83us) server_negotiation.cc:407] Connection header received
0925 02:43:00.745124 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:00.745128 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:00.745220 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:00.745294 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:00.745975 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:00.746480 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:00.747363 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:00.747586 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:00.747675 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:00.750076 (+ 2401us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:00.750080 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:00.750083 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:43:00.750108 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:00.751374 (+ 1266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:00.751780 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:00.751783 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:00.751785 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:00.751864 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:00.752106 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:00.752109 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:00.752111 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:00.752295 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:00.752413 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:00.752655 (+ 242us) server_negotiation.cc:299] Negotiation successful
0925 02:43:00.752781 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260925 02:43:00.754096 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 14056
I20260925 02:43:00.759944 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 14190
I20260925 02:43:00.765333 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 14324
I20260925 02:43:00.770219 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 13962
2026-09-25T02:43:00Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2290 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790304178325258-13942-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 02:43:00 dist-test-slave-kznw krb5kdc[14467](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 02:43:00 dist-test-slave-kznw krb5kdc[14467](info): set up 2 sockets
Sep 25 02:43:00 dist-test-slave-kznw krb5kdc[14467](info): commencing operation
krb5kdc: starting...
W20260925 02:43:01.726142 13942 mini_kdc.cc:121] Time spent starting KDC: real 0.934s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790304178325258-13942-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790304178325258-13942-0/krb5kdc/test-user.keytab.
Sep 25 02:43:01 dist-test-slave-kznw krb5kdc[14467](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304181, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T02:43:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T02:43:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.157.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.157.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.157.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab.
Entry for principal kudu/127.13.157.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab.
I20260925 02:43:01.845768 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790304178325258-13942-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790304178325258-13942-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790304178325258-13942-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.157.190:33727
--webserver_interface=127.13.157.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab
--principal=kudu/127.13.157.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:41245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.157.190:33727
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:43:01.955386 14480 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:43:01.955675 14480 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:43:01.955720 14480 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:43:01.959772 14480 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:43:01.959841 14480 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:43:01.959859 14480 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:43:01.959874 14480 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:43:01.959889 14480 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:43:01.965173 14480 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:41245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790304178325258-13942-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790304178325258-13942-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.157.190:33727
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab
--principal=kudu/127.13.157.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.157.190:33727
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.157.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14480
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790304178325258-13942-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:43:01.966217 14480 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:43:01.967257 14480 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:43:01.975849 14485 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:01.975857 14488 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:43:01.975934 14480 server_base.cc:1034] running on GCE node
W20260925 02:43:01.976089 14486 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:43:01.976680 14480 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:43:01.977748 14480 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:43:01.978935 14480 hybrid_clock.cc:648] HybridClock initialized: now 1790304181978894 us; error 31 us; skew 500 ppm
Sep 25 02:43:01 dist-test-slave-kznw krb5kdc[14467](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304181, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:43:01.981734 14480 init.cc:377] Logged in from keytab as kudu/127.13.157.190@KRBTEST.COM (short username kudu)
I20260925 02:43:01.982663 14480 webserver.cc:460] Webserver started at http://127.13.157.190:33367/ using document root <none> and password file <none>
I20260925 02:43:01.983244 14480 fs_manager.cc:359] Metadata directory not provided
I20260925 02:43:01.983314 14480 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:43:01.983518 14480 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:43:01.985526 14480 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790304178325258-13942-0/minicluster-data/master-0/data/instance:
uuid: "61404e81d71441449bfb09bf294f5e7b"
format_stamp: "Formatted at 2026-09-25 02:43:01 on dist-test-slave-kznw"
server_key: "f6a82c3c3115c38f6f3a78f772adc48f"
server_key_iv: "0997775389c90d56feb605000bf5704d"
server_key_version: "encryptionkey@0"
I20260925 02:43:01.986068 14480 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790304178325258-13942-0/minicluster-data/master-0/wal/instance:
uuid: "61404e81d71441449bfb09bf294f5e7b"
format_stamp: "Formatted at 2026-09-25 02:43:01 on dist-test-slave-kznw"
server_key: "f6a82c3c3115c38f6f3a78f772adc48f"
server_key_iv: "0997775389c90d56feb605000bf5704d"
server_key_version: "encryptionkey@0"
I20260925 02:43:01.989696 14480 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260925 02:43:01.992157 14495 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:01.993355 14480 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260925 02:43:01.993479 14480 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790304178325258-13942-0/minicluster-data/master-0/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790304178325258-13942-0/minicluster-data/master-0/wal
uuid: "61404e81d71441449bfb09bf294f5e7b"
format_stamp: "Formatted at 2026-09-25 02:43:01 on dist-test-slave-kznw"
server_key: "f6a82c3c3115c38f6f3a78f772adc48f"
server_key_iv: "0997775389c90d56feb605000bf5704d"
server_key_version: "encryptionkey@0"
I20260925 02:43:01.993570 14480 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790304178325258-13942-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790304178325258-13942-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790304178325258-13942-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:43:02.012229 14480 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:43:02.015368 14480 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:43:02.015545 14480 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:43:02.022313 14480 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.190:33727
I20260925 02:43:02.022339 14547 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.190:33727 every 8 connection(s)
I20260925 02:43:02.023298 14480 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
I20260925 02:43:02.026087 14548 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:43:02.031595 14548 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61404e81d71441449bfb09bf294f5e7b: Bootstrap starting.
I20260925 02:43:02.031960 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 14480
I20260925 02:43:02.032073 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790304178325258-13942-0/minicluster-data/master-0/wal/instance
I20260925 02:43:02.032326 13942 external_mini_cluster.cc:1428] Setting key dc8206161b3fe9a5451052dd5887eea5
I20260925 02:43:02.034298 14548 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 61404e81d71441449bfb09bf294f5e7b: Neither blocks nor log segments found. Creating new log.
I20260925 02:43:02.035012 14548 log.cc:824] T 00000000000000000000000000000000 P 61404e81d71441449bfb09bf294f5e7b: Log is configured to *not* fsync() on all Append() calls
I20260925 02:43:02.037164 14548 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61404e81d71441449bfb09bf294f5e7b: No bootstrap required, opened a new log
Sep 25 02:43:02 dist-test-slave-kznw krb5kdc[14467](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304181, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.157.190@KRBTEST.COM
I20260925 02:43:02.042877 14548 raft_consensus.cc:348] T 00000000000000000000000000000000 P 61404e81d71441449bfb09bf294f5e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61404e81d71441449bfb09bf294f5e7b" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 33727 } }
I20260925 02:43:02.043200 14548 raft_consensus.cc:374] T 00000000000000000000000000000000 P 61404e81d71441449bfb09bf294f5e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:43:02.043272 14548 raft_consensus.cc:729] T 00000000000000000000000000000000 P 61404e81d71441449bfb09bf294f5e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61404e81d71441449bfb09bf294f5e7b, State: Initialized, Role: FOLLOWER
I20260925 02:43:02.043789 14548 consensus_queue.cc:260] T 00000000000000000000000000000000 P 61404e81d71441449bfb09bf294f5e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61404e81d71441449bfb09bf294f5e7b" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 33727 } }
I20260925 02:43:02.043953 14548 raft_consensus.cc:388] T 00000000000000000000000000000000 P 61404e81d71441449bfb09bf294f5e7b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 02:43:02.044026 14548 raft_consensus.cc:482] T 00000000000000000000000000000000 P 61404e81d71441449bfb09bf294f5e7b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 02:43:02.044103 14548 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 61404e81d71441449bfb09bf294f5e7b [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:43:02.045149 14548 raft_consensus.cc:504] T 00000000000000000000000000000000 P 61404e81d71441449bfb09bf294f5e7b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61404e81d71441449bfb09bf294f5e7b" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 33727 } }
I20260925 02:43:02.045509 14548 leader_election.cc:302] T 00000000000000000000000000000000 P 61404e81d71441449bfb09bf294f5e7b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 61404e81d71441449bfb09bf294f5e7b; no voters:
I20260925 02:43:02.045888 14548 leader_election.cc:288] T 00000000000000000000000000000000 P 61404e81d71441449bfb09bf294f5e7b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 02:43:02.046036 14553 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 61404e81d71441449bfb09bf294f5e7b [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:43:02.046232 14553 raft_consensus.cc:686] T 00000000000000000000000000000000 P 61404e81d71441449bfb09bf294f5e7b [term 1 LEADER]: Becoming Leader. State: Replica: 61404e81d71441449bfb09bf294f5e7b, State: Running, Role: LEADER
I20260925 02:43:02.046584 14553 consensus_queue.cc:237] T 00000000000000000000000000000000 P 61404e81d71441449bfb09bf294f5e7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61404e81d71441449bfb09bf294f5e7b" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 33727 } }
I20260925 02:43:02.047050 14551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:02.033736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59558 (local address 127.13.157.190:33727)
0925 02:43:02.034214 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:02.034231 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:02.034268 (+ 37us) server_negotiation.cc:407] Connection header received
0925 02:43:02.034980 (+ 712us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:02.035000 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:02.035382 (+ 382us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:02.035757 (+ 375us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:02.036649 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:02.037679 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:02.038824 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:02.039335 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:02.039457 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:02.041853 (+ 2396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:02.041860 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:02.041876 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:43:02.041907 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:02.043878 (+ 1971us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:02.044389 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:02.044395 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:02.044401 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:02.044537 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:02.044768 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:02.044770 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:02.044772 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:02.045311 (+ 539us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:02.045487 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:02.045740 (+ 253us) server_negotiation.cc:299] Negotiation successful
0925 02:43:02.046041 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":292,"thread_start_us":120,"threads_started":1}
I20260925 02:43:02.047189 14548 sys_catalog.cc:564] T 00000000000000000000000000000000 P 61404e81d71441449bfb09bf294f5e7b [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:43:02.048210 14554 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61404e81d71441449bfb09bf294f5e7b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "61404e81d71441449bfb09bf294f5e7b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61404e81d71441449bfb09bf294f5e7b" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 33727 } } }
I20260925 02:43:02.048337 14554 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61404e81d71441449bfb09bf294f5e7b [sys.catalog]: This master's current role is: LEADER
I20260925 02:43:02.048605 14555 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61404e81d71441449bfb09bf294f5e7b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 61404e81d71441449bfb09bf294f5e7b. Latest consensus state: current_term: 1 leader_uuid: "61404e81d71441449bfb09bf294f5e7b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61404e81d71441449bfb09bf294f5e7b" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 33727 } } }
I20260925 02:43:02.048691 14555 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61404e81d71441449bfb09bf294f5e7b [sys.catalog]: This master's current role is: LEADER
I20260925 02:43:02.049307 14562 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:43:02.052604 14562 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:43:02.058413 14562 catalog_manager.cc:1347] Generated new cluster ID: a590a9a0c4704c898320660e6e8350b9
I20260925 02:43:02.058476 14562 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 02:43:02.080286 14562 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 02:43:02.081626 14562 catalog_manager.cc:1504] Loading token signing keys...
I20260925 02:43:02.088673 14562 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 61404e81d71441449bfb09bf294f5e7b: Generated new TSK 0
I20260925 02:43:02.089373 14562 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.157.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.157.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.157.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.129.keytab.
Entry for principal kudu/127.13.157.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.129.keytab.
I20260925 02:43:02.129391 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790304178325258-13942-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790304178325258-13942-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790304178325258-13942-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790304178325258-13942-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.157.129:0
--local_ip_for_outbound_sockets=127.13.157.129
--webserver_interface=127.13.157.129
--webserver_port=0
--tserver_master_addrs=127.13.157.190:33727
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.129.keytab
--principal=kudu/127.13.157.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:41245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:43:02.239457 14574 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:43:02.239723 14574 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:43:02.239768 14574 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:43:02.243746 14574 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:43:02.243803 14574 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:43:02.243860 14574 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.157.129
I20260925 02:43:02.249691 14574 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:41245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790304178325258-13942-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790304178325258-13942-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.129.keytab
--principal=kudu/127.13.157.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.157.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790304178325258-13942-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.157.129
--webserver_port=0
--tserver_master_addrs=127.13.157.190:33727
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14574
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.157.129
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790304178325258-13942-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:43:02.250768 14574 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:43:02.251809 14574 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:43:02.261833 14582 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:02.261825 14579 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:43:02.261884 14574 server_base.cc:1034] running on GCE node
W20260925 02:43:02.261840 14580 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:43:02.262434 14574 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:43:02.263118 14574 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:43:02.264286 14574 hybrid_clock.cc:648] HybridClock initialized: now 1790304182264266 us; error 32 us; skew 500 ppm
Sep 25 02:43:02 dist-test-slave-kznw krb5kdc[14467](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304182, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:43:02.266978 14574 init.cc:377] Logged in from keytab as kudu/127.13.157.129@KRBTEST.COM (short username kudu)
I20260925 02:43:02.267940 14574 webserver.cc:460] Webserver started at http://127.13.157.129:40513/ using document root <none> and password file <none>
I20260925 02:43:02.268451 14574 fs_manager.cc:359] Metadata directory not provided
I20260925 02:43:02.268507 14574 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:43:02.268673 14574 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:43:02.270643 14574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790304178325258-13942-0/minicluster-data/ts-0/data/instance:
uuid: "cc2903be8270464099694a1d759b63f5"
format_stamp: "Formatted at 2026-09-25 02:43:02 on dist-test-slave-kznw"
server_key: "b29d0c24301026de8b91b527618a4ea1"
server_key_iv: "f4382dd64b3852e4a9c966fe7c82f50a"
server_key_version: "encryptionkey@0"
I20260925 02:43:02.271201 14574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790304178325258-13942-0/minicluster-data/ts-0/wal/instance:
uuid: "cc2903be8270464099694a1d759b63f5"
format_stamp: "Formatted at 2026-09-25 02:43:02 on dist-test-slave-kznw"
server_key: "b29d0c24301026de8b91b527618a4ea1"
server_key_iv: "f4382dd64b3852e4a9c966fe7c82f50a"
server_key_version: "encryptionkey@0"
I20260925 02:43:02.274705 14574 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260925 02:43:02.277024 14589 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:02.278210 14574 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260925 02:43:02.278321 14574 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790304178325258-13942-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790304178325258-13942-0/minicluster-data/ts-0/wal
uuid: "cc2903be8270464099694a1d759b63f5"
format_stamp: "Formatted at 2026-09-25 02:43:02 on dist-test-slave-kznw"
server_key: "b29d0c24301026de8b91b527618a4ea1"
server_key_iv: "f4382dd64b3852e4a9c966fe7c82f50a"
server_key_version: "encryptionkey@0"
I20260925 02:43:02.278420 14574 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790304178325258-13942-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790304178325258-13942-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790304178325258-13942-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:43:02.293478 14574 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:43:02.296492 14574 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:43:02.296680 14574 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:43:02.297272 14574 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:43:02.298101 14574 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:43:02.298146 14574 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:02.298190 14574 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:43:02.298218 14574 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:02.306886 14574 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.129:34087
I20260925 02:43:02.306957 14703 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.129:34087 every 8 connection(s)
I20260925 02:43:02.307813 14574 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790304178325258-13942-0/minicluster-data/ts-0/data/info.pb
I20260925 02:43:02.315428 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 14574
I20260925 02:43:02.315522 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790304178325258-13942-0/minicluster-data/ts-0/wal/instance
I20260925 02:43:02.315757 13942 external_mini_cluster.cc:1428] Setting key 98b7260e1a3a0cf4a1bb9f0d4ba0648b
Sep 25 02:43:02 dist-test-slave-kznw krb5kdc[14467](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304182, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.129@KRBTEST.COM for kudu/127.13.157.190@KRBTEST.COM
I20260925 02:43:02.321630 14551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:02.309527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.129:32823 (local address 127.13.157.190:33727)
0925 02:43:02.309671 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:02.309674 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:02.310589 (+ 915us) server_negotiation.cc:407] Connection header received
0925 02:43:02.311892 (+ 1303us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:02.311896 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:02.311969 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:02.312042 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:02.313183 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:02.313719 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:02.314836 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:02.315119 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:02.315210 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:02.317820 (+ 2610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:02.317824 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:02.317827 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:43:02.317856 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:02.319374 (+ 1518us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:02.320013 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:02.320016 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:02.320017 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:02.320113 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:02.320479 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:02.320482 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:02.320483 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:02.320684 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:02.320778 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:02.321361 (+ 583us) server_negotiation.cc:299] Negotiation successful
0925 02:43:02.321490 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260925 02:43:02.322775 14706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:02.309913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:33727 (local address 127.13.157.129:32823)
0925 02:43:02.310438 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:02.310481 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:02.311691 (+ 1210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:02.312225 (+ 534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:02.312232 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:02.312579 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:02.313042 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:02.313055 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:02.313836 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:02.313838 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:02.314718 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:02.314728 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:02.315270 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:02.315274 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:02.315437 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:02.315996 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:02.316011 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:02.317648 (+ 1637us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:43:02.319492 (+ 1844us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:02.319498 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:02.319514 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:02.319900 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:02.320211 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:02.320214 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:02.320216 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:02.320383 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:02.320788 (+ 405us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:43:02.320794 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:43:02.321089 (+ 295us) client_negotiation.cc:769] Sending connection context
0925 02:43:02.321298 (+ 209us) client_negotiation.cc:240] Negotiation successful
0925 02:43:02.321647 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":118,"threads_started":1}
I20260925 02:43:02.324044 14704 heartbeater.cc:347] Connected to a master server at 127.13.157.190:33727
I20260925 02:43:02.324266 14704 heartbeater.cc:464] Registering TS with master...
I20260925 02:43:02.324736 14704 heartbeater.cc:511] Master 127.13.157.190:33727 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.13.157.130@KRBTEST.COM; defaulting to no policy
I20260925 02:43:02.326220 14512 ts_manager.cc:194] Registered new tserver with Master: cc2903be8270464099694a1d759b63f5 (127.13.157.129:34087)
I20260925 02:43:02.327432 14512 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.157.129@KRBTEST.COM'} at 127.13.157.129:32823
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.157.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.157.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.130.keytab.
Entry for principal kudu/127.13.157.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.130.keytab.
I20260925 02:43:02.342038 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790304178325258-13942-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790304178325258-13942-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790304178325258-13942-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790304178325258-13942-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.157.130:0
--local_ip_for_outbound_sockets=127.13.157.130
--webserver_interface=127.13.157.130
--webserver_port=0
--tserver_master_addrs=127.13.157.190:33727
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.130.keytab
--principal=kudu/127.13.157.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:41245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:43:02.448750 14709 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:43:02.449002 14709 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:43:02.449071 14709 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:43:02.453048 14709 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:43:02.453110 14709 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:43:02.453189 14709 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.157.130
I20260925 02:43:02.458590 14709 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:41245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790304178325258-13942-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790304178325258-13942-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.130.keytab
--principal=kudu/127.13.157.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.157.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790304178325258-13942-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.157.130
--webserver_port=0
--tserver_master_addrs=127.13.157.190:33727
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14709
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.157.130
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790304178325258-13942-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:43:02.459663 14709 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:43:02.460673 14709 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:43:02.469798 14717 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:02.469811 14715 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:02.469862 14714 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:43:02.469969 14709 server_base.cc:1034] running on GCE node
I20260925 02:43:02.470422 14709 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:43:02.471062 14709 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:43:02.472209 14709 hybrid_clock.cc:648] HybridClock initialized: now 1790304182472193 us; error 59 us; skew 500 ppm
Sep 25 02:43:02 dist-test-slave-kznw krb5kdc[14467](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304182, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:43:02.474714 14709 init.cc:377] Logged in from keytab as kudu/127.13.157.130@KRBTEST.COM (short username kudu)
I20260925 02:43:02.475553 14709 webserver.cc:460] Webserver started at http://127.13.157.130:35737/ using document root <none> and password file <none>
I20260925 02:43:02.476063 14709 fs_manager.cc:359] Metadata directory not provided
I20260925 02:43:02.476126 14709 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:43:02.476326 14709 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:43:02.478256 14709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790304178325258-13942-0/minicluster-data/ts-1/data/instance:
uuid: "037a824541264914af1453e1a583f168"
format_stamp: "Formatted at 2026-09-25 02:43:02 on dist-test-slave-kznw"
server_key: "39c4d3cb16db539a2f909caac304aea1"
server_key_iv: "7e452a3cd7592b6f2457fbd36fe6a9b4"
server_key_version: "encryptionkey@0"
I20260925 02:43:02.478789 14709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790304178325258-13942-0/minicluster-data/ts-1/wal/instance:
uuid: "037a824541264914af1453e1a583f168"
format_stamp: "Formatted at 2026-09-25 02:43:02 on dist-test-slave-kznw"
server_key: "39c4d3cb16db539a2f909caac304aea1"
server_key_iv: "7e452a3cd7592b6f2457fbd36fe6a9b4"
server_key_version: "encryptionkey@0"
I20260925 02:43:02.482414 14709 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.002s
I20260925 02:43:02.484793 14724 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:02.486017 14709 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260925 02:43:02.486138 14709 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790304178325258-13942-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790304178325258-13942-0/minicluster-data/ts-1/wal
uuid: "037a824541264914af1453e1a583f168"
format_stamp: "Formatted at 2026-09-25 02:43:02 on dist-test-slave-kznw"
server_key: "39c4d3cb16db539a2f909caac304aea1"
server_key_iv: "7e452a3cd7592b6f2457fbd36fe6a9b4"
server_key_version: "encryptionkey@0"
I20260925 02:43:02.486274 14709 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790304178325258-13942-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790304178325258-13942-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790304178325258-13942-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:43:02.496516 14709 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:43:02.499586 14709 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:43:02.499739 14709 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:43:02.500260 14709 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:43:02.501020 14709 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:43:02.501063 14709 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:02.501101 14709 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:43:02.501119 14709 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:02.508986 14709 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.130:41685
I20260925 02:43:02.509014 14839 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.130:41685 every 8 connection(s)
I20260925 02:43:02.509826 14709 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790304178325258-13942-0/minicluster-data/ts-1/data/info.pb
I20260925 02:43:02.517625 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 14709
I20260925 02:43:02.517756 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790304178325258-13942-0/minicluster-data/ts-1/wal/instance
I20260925 02:43:02.518005 13942 external_mini_cluster.cc:1428] Setting key 13eef9e13cf179b005bab680e92e848b
Sep 25 02:43:02 dist-test-slave-kznw krb5kdc[14467](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304182, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.130@KRBTEST.COM for kudu/127.13.157.190@KRBTEST.COM
I20260925 02:43:02.522641 14551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:02.511499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.130:56949 (local address 127.13.157.190:33727)
0925 02:43:02.511632 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:02.511637 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:02.512503 (+ 866us) server_negotiation.cc:407] Connection header received
0925 02:43:02.513709 (+ 1206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:02.513712 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:02.513757 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:02.513827 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:02.514898 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:02.515449 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:02.516520 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:02.516742 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:02.516823 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:02.519288 (+ 2465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:02.519291 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:02.519293 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:43:02.519317 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:02.520688 (+ 1371us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:02.521207 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:02.521210 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:02.521212 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:02.521292 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:02.521602 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:02.521605 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:02.521607 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:02.521792 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:02.521877 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:02.522353 (+ 476us) server_negotiation.cc:299] Negotiation successful
0925 02:43:02.522474 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260925 02:43:02.523538 14842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:02.511911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:33727 (local address 127.13.157.130:56949)
0925 02:43:02.512361 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:02.512403 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:02.513528 (+ 1125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:02.513971 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:02.513978 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:02.514306 (+ 328us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:02.514783 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:02.514796 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:02.515549 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:02.515552 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:02.516405 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:02.516414 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:02.516860 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:02.516863 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:02.516996 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:02.517529 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:02.517544 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:02.519135 (+ 1591us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:43:02.520835 (+ 1700us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:02.520840 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:02.520851 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:02.521092 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:02.521376 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:02.521380 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:02.521381 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:02.521514 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:02.521889 (+ 375us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:43:02.521895 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:43:02.522113 (+ 218us) client_negotiation.cc:769] Sending connection context
0925 02:43:02.522295 (+ 182us) client_negotiation.cc:240] Negotiation successful
0925 02:43:02.522600 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":127,"threads_started":1}
I20260925 02:43:02.524735 14840 heartbeater.cc:347] Connected to a master server at 127.13.157.190:33727
I20260925 02:43:02.524991 14840 heartbeater.cc:464] Registering TS with master...
I20260925 02:43:02.525564 14840 heartbeater.cc:511] Master 127.13.157.190:33727 requested a full tablet report, sending...
I20260925 02:43:02.526691 14512 ts_manager.cc:194] Registered new tserver with Master: 037a824541264914af1453e1a583f168 (127.13.157.130:41685)
WARNING: no policy specified for kudu/127.13.157.131@KRBTEST.COM; defaulting to no policy
I20260925 02:43:02.527315 14512 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.157.130@KRBTEST.COM'} at 127.13.157.130:56949
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.157.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.157.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.131.keytab.
Entry for principal kudu/127.13.157.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.131.keytab.
I20260925 02:43:02.544023 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790304178325258-13942-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790304178325258-13942-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790304178325258-13942-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790304178325258-13942-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.157.131:0
--local_ip_for_outbound_sockets=127.13.157.131
--webserver_interface=127.13.157.131
--webserver_port=0
--tserver_master_addrs=127.13.157.190:33727
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.131.keytab
--principal=kudu/127.13.157.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:41245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:43:02.654951 14845 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:43:02.655247 14845 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:43:02.655297 14845 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:43:02.659468 14845 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:43:02.659525 14845 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:43:02.659596 14845 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.157.131
I20260925 02:43:02.664979 14845 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:41245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790304178325258-13942-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790304178325258-13942-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.131.keytab
--principal=kudu/127.13.157.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.157.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790304178325258-13942-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.157.131
--webserver_port=0
--tserver_master_addrs=127.13.157.190:33727
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14845
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.157.131
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790304178325258-13942-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:43:02.666097 14845 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:43:02.667184 14845 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:43:02.676712 14853 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:02.676801 14850 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:43:02.676868 14845 server_base.cc:1034] running on GCE node
W20260925 02:43:02.676802 14851 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:43:02.677340 14845 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:43:02.677968 14845 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:43:02.679119 14845 hybrid_clock.cc:648] HybridClock initialized: now 1790304182679102 us; error 27 us; skew 500 ppm
Sep 25 02:43:02 dist-test-slave-kznw krb5kdc[14467](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304182, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:43:02.681828 14845 init.cc:377] Logged in from keytab as kudu/127.13.157.131@KRBTEST.COM (short username kudu)
I20260925 02:43:02.682817 14845 webserver.cc:460] Webserver started at http://127.13.157.131:33427/ using document root <none> and password file <none>
I20260925 02:43:02.683388 14845 fs_manager.cc:359] Metadata directory not provided
I20260925 02:43:02.683458 14845 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:43:02.683658 14845 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:43:02.685667 14845 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790304178325258-13942-0/minicluster-data/ts-2/data/instance:
uuid: "2079ff48d01f423b81313343b63c2be5"
format_stamp: "Formatted at 2026-09-25 02:43:02 on dist-test-slave-kznw"
server_key: "628022b95a7f990c512efcb54f32f8f7"
server_key_iv: "c7a71cd4f11262eed6076abd1fc75cfb"
server_key_version: "encryptionkey@0"
I20260925 02:43:02.686203 14845 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790304178325258-13942-0/minicluster-data/ts-2/wal/instance:
uuid: "2079ff48d01f423b81313343b63c2be5"
format_stamp: "Formatted at 2026-09-25 02:43:02 on dist-test-slave-kznw"
server_key: "628022b95a7f990c512efcb54f32f8f7"
server_key_iv: "c7a71cd4f11262eed6076abd1fc75cfb"
server_key_version: "encryptionkey@0"
I20260925 02:43:02.689778 14845 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260925 02:43:02.692231 14860 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:02.693379 14845 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260925 02:43:02.693482 14845 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790304178325258-13942-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790304178325258-13942-0/minicluster-data/ts-2/wal
uuid: "2079ff48d01f423b81313343b63c2be5"
format_stamp: "Formatted at 2026-09-25 02:43:02 on dist-test-slave-kznw"
server_key: "628022b95a7f990c512efcb54f32f8f7"
server_key_iv: "c7a71cd4f11262eed6076abd1fc75cfb"
server_key_version: "encryptionkey@0"
I20260925 02:43:02.693571 14845 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790304178325258-13942-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790304178325258-13942-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790304178325258-13942-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:43:02.705150 14845 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:43:02.708194 14845 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:43:02.708372 14845 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:43:02.708950 14845 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:43:02.709789 14845 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:43:02.709836 14845 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:02.709887 14845 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:43:02.709913 14845 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:02.718142 14845 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.131:33921
I20260925 02:43:02.718223 14974 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.131:33921 every 8 connection(s)
I20260925 02:43:02.719090 14845 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790304178325258-13942-0/minicluster-data/ts-2/data/info.pb
I20260925 02:43:02.720484 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 14845
I20260925 02:43:02.720575 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790304178325258-13942-0/minicluster-data/ts-2/wal/instance
I20260925 02:43:02.720844 13942 external_mini_cluster.cc:1428] Setting key 48aa08937055b3267b04d69f6518d2dd
Sep 25 02:43:02 dist-test-slave-kznw krb5kdc[14467](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304182, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.131@KRBTEST.COM for kudu/127.13.157.190@KRBTEST.COM
I20260925 02:43:02.732213 14551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:02.720880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.131:56927 (local address 127.13.157.190:33727)
0925 02:43:02.721029 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:02.721034 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:02.722032 (+ 998us) server_negotiation.cc:407] Connection header received
0925 02:43:02.723545 (+ 1513us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:02.723548 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:02.723603 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:02.723672 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:02.724837 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:02.725407 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:02.726524 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:02.726741 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:02.726831 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:02.729111 (+ 2280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:02.729114 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:02.729116 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:43:02.729141 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:02.730278 (+ 1137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:02.730737 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:02.730740 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:02.730741 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:02.730817 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:02.731145 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:02.731148 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:02.731150 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:02.731332 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:02.731421 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:02.731913 (+ 492us) server_negotiation.cc:299] Negotiation successful
0925 02:43:02.732034 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260925 02:43:02.732995 14977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:02.721375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:33727 (local address 127.13.157.131:56927)
0925 02:43:02.721884 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:02.721942 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:02.723344 (+ 1402us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:02.723861 (+ 517us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:02.723868 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:02.724212 (+ 344us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:02.724695 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:02.724709 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:02.725524 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:02.725528 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:02.726405 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:02.726414 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:02.726878 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:02.726882 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:02.727070 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:02.727555 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:02.727570 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:02.728950 (+ 1380us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:43:02.730384 (+ 1434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:02.730390 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:02.730405 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:02.730637 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:02.730896 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:02.730899 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:02.730901 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:02.731058 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:02.731430 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:43:02.731436 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:43:02.731682 (+ 246us) client_negotiation.cc:769] Sending connection context
0925 02:43:02.731839 (+ 157us) client_negotiation.cc:240] Negotiation successful
0925 02:43:02.732148 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":129,"threads_started":1}
I20260925 02:43:02.734310 14975 heartbeater.cc:347] Connected to a master server at 127.13.157.190:33727
I20260925 02:43:02.734539 14975 heartbeater.cc:464] Registering TS with master...
I20260925 02:43:02.735128 14975 heartbeater.cc:511] Master 127.13.157.190:33727 requested a full tablet report, sending...
I20260925 02:43:02.736266 14512 ts_manager.cc:194] Registered new tserver with Master: 2079ff48d01f423b81313343b63c2be5 (127.13.157.131:33921)
I20260925 02:43:02.736830 14512 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.157.131@KRBTEST.COM'} at 127.13.157.131:56927
I20260925 02:43:02.746776 13942 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 02:43:02.758652 14551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:02.751400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59564 (local address 127.13.157.190:33727)
0925 02:43:02.751537 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:02.751540 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:02.751732 (+ 192us) server_negotiation.cc:407] Connection header received
0925 02:43:02.751854 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:02.751857 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:02.751900 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:02.751958 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:02.752680 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:02.753204 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:02.754076 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:02.754307 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:02.754390 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:02.755646 (+ 1256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:02.755649 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:02.755651 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:43:02.755677 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:02.756973 (+ 1296us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:02.757382 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:02.757385 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:02.757387 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:02.757487 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:02.757813 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:02.757817 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:02.757818 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:02.758052 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:02.758157 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:02.758397 (+ 240us) server_negotiation.cc:299] Negotiation successful
0925 02:43:02.758493 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260925 02:43:02.763582 14512 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59564:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 02:43:02.766570 14512 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 02:43:02.780313 14987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:02.774772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33530 (local address 127.13.157.129:34087)
0925 02:43:02.775258 (+ 486us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:02.775269 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:02.775294 (+ 25us) server_negotiation.cc:407] Connection header received
0925 02:43:02.775481 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:02.775487 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:02.775722 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:02.775899 (+ 177us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:43:02.776418 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:02.777361 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:02.778757 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:02.779625 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:02.779744 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:02.779981 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:02.780054 (+ 73us) server_negotiation.cc:299] Negotiation successful
0925 02:43:02.780155 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":386,"thread_start_us":166,"threads_started":1}
I20260925 02:43:02.780309 14986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:02.774639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.129:34087 (local address 127.0.0.1:33530)
0925 02:43:02.775036 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:02.775154 (+ 118us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:02.775285 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:02.776112 (+ 827us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:02.776119 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:02.776145 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:43:02.776287 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:02.776295 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:02.777463 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:02.777466 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:02.778603 (+ 1137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:02.778613 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:02.779742 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:02.779746 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:02.779875 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:02.779920 (+ 45us) client_negotiation.cc:769] Sending connection context
0925 02:43:02.780021 (+ 101us) client_negotiation.cc:240] Negotiation successful
0925 02:43:02.780143 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":99,"threads_started":1}
I20260925 02:43:02.780580 14989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:02.775050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.130:41685 (local address 127.0.0.1:58074)
0925 02:43:02.775671 (+ 621us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:02.775684 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:02.775787 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:02.776340 (+ 553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:02.776344 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:02.776361 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:43:02.776457 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:02.776463 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:02.777989 (+ 1526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:02.777993 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:02.779190 (+ 1197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:02.779201 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:02.780237 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:02.780241 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:02.780324 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:02.780346 (+ 22us) client_negotiation.cc:769] Sending connection context
0925 02:43:02.780392 (+ 46us) client_negotiation.cc:240] Negotiation successful
0925 02:43:02.780458 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":563,"mutex_wait_us":42,"thread_start_us":88,"threads_started":1}
I20260925 02:43:02.780869 14988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:02.775407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.131:33921 (local address 127.0.0.1:48892)
0925 02:43:02.775947 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:02.775958 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:02.776028 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:02.776887 (+ 859us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:02.776890 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:02.776909 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:43:02.776986 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:02.776992 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:02.778417 (+ 1425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:02.778421 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:02.779410 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:02.779419 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:02.780529 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:02.780533 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:02.780613 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:02.780634 (+ 21us) client_negotiation.cc:769] Sending connection context
0925 02:43:02.780680 (+ 46us) client_negotiation.cc:240] Negotiation successful
0925 02:43:02.780737 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":488,"mutex_wait_us":1,"thread_start_us":77,"threads_started":1}
I20260925 02:43:02.781183 14990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:02.775482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58074 (local address 127.13.157.130:41685)
0925 02:43:02.775881 (+ 399us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:02.775893 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:02.775914 (+ 21us) server_negotiation.cc:407] Connection header received
0925 02:43:02.775988 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:02.775993 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:02.776183 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:02.776611 (+ 428us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:43:02.776763 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:02.777850 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:02.779320 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:02.780108 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:02.780836 (+ 728us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:02.780928 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:02.780977 (+ 49us) server_negotiation.cc:299] Negotiation successful
0925 02:43:02.781059 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":131,"threads_started":1}
I20260925 02:43:02.781427 14991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:02.775714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48892 (local address 127.13.157.131:33921)
0925 02:43:02.776273 (+ 559us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:02.776285 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:02.776306 (+ 21us) server_negotiation.cc:407] Connection header received
0925 02:43:02.776373 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:02.776378 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:02.776737 (+ 359us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:02.776891 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:43:02.777097 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:02.778250 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:02.779516 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:02.780267 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:02.781052 (+ 785us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:02.781169 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:02.781213 (+ 44us) server_negotiation.cc:299] Negotiation successful
0925 02:43:02.781293 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":461,"thread_start_us":137,"threads_started":1}
I20260925 02:43:02.783830 14773 tablet_service.cc:1467] Processing CreateTablet for tablet c3de2f7c6a694cfa9add33b164f548cd (DEFAULT_TABLE table=test-table [id=e5110b542f694a419f8cb2a440114b62]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:43:02.783830 14637 tablet_service.cc:1467] Processing CreateTablet for tablet c3de2f7c6a694cfa9add33b164f548cd (DEFAULT_TABLE table=test-table [id=e5110b542f694a419f8cb2a440114b62]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:43:02.784986 14773 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c3de2f7c6a694cfa9add33b164f548cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:43:02.784986 14637 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c3de2f7c6a694cfa9add33b164f548cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:43:02.785480 14908 tablet_service.cc:1467] Processing CreateTablet for tablet c3de2f7c6a694cfa9add33b164f548cd (DEFAULT_TABLE table=test-table [id=e5110b542f694a419f8cb2a440114b62]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:43:02.786285 14908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c3de2f7c6a694cfa9add33b164f548cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:43:02.793159 14994 tablet_bootstrap.cc:492] T c3de2f7c6a694cfa9add33b164f548cd P 2079ff48d01f423b81313343b63c2be5: Bootstrap starting.
I20260925 02:43:02.793323 14992 tablet_bootstrap.cc:492] T c3de2f7c6a694cfa9add33b164f548cd P cc2903be8270464099694a1d759b63f5: Bootstrap starting.
I20260925 02:43:02.795233 14993 tablet_bootstrap.cc:492] T c3de2f7c6a694cfa9add33b164f548cd P 037a824541264914af1453e1a583f168: Bootstrap starting.
I20260925 02:43:02.795310 14994 tablet_bootstrap.cc:654] T c3de2f7c6a694cfa9add33b164f548cd P 2079ff48d01f423b81313343b63c2be5: Neither blocks nor log segments found. Creating new log.
I20260925 02:43:02.795361 14992 tablet_bootstrap.cc:654] T c3de2f7c6a694cfa9add33b164f548cd P cc2903be8270464099694a1d759b63f5: Neither blocks nor log segments found. Creating new log.
I20260925 02:43:02.797539 14993 tablet_bootstrap.cc:654] T c3de2f7c6a694cfa9add33b164f548cd P 037a824541264914af1453e1a583f168: Neither blocks nor log segments found. Creating new log.
I20260925 02:43:02.797919 14994 log.cc:824] T c3de2f7c6a694cfa9add33b164f548cd P 2079ff48d01f423b81313343b63c2be5: Log is configured to *not* fsync() on all Append() calls
I20260925 02:43:02.798074 14992 log.cc:824] T c3de2f7c6a694cfa9add33b164f548cd P cc2903be8270464099694a1d759b63f5: Log is configured to *not* fsync() on all Append() calls
I20260925 02:43:02.798840 14993 log.cc:824] T c3de2f7c6a694cfa9add33b164f548cd P 037a824541264914af1453e1a583f168: Log is configured to *not* fsync() on all Append() calls
I20260925 02:43:02.800359 14992 tablet_bootstrap.cc:492] T c3de2f7c6a694cfa9add33b164f548cd P cc2903be8270464099694a1d759b63f5: No bootstrap required, opened a new log
I20260925 02:43:02.800561 14992 ts_tablet_manager.cc:1397] T c3de2f7c6a694cfa9add33b164f548cd P cc2903be8270464099694a1d759b63f5: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260925 02:43:02.800932 14993 tablet_bootstrap.cc:492] T c3de2f7c6a694cfa9add33b164f548cd P 037a824541264914af1453e1a583f168: No bootstrap required, opened a new log
I20260925 02:43:02.801119 14993 ts_tablet_manager.cc:1397] T c3de2f7c6a694cfa9add33b164f548cd P 037a824541264914af1453e1a583f168: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260925 02:43:02.801620 14994 tablet_bootstrap.cc:492] T c3de2f7c6a694cfa9add33b164f548cd P 2079ff48d01f423b81313343b63c2be5: No bootstrap required, opened a new log
I20260925 02:43:02.801781 14994 ts_tablet_manager.cc:1397] T c3de2f7c6a694cfa9add33b164f548cd P 2079ff48d01f423b81313343b63c2be5: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.000s
I20260925 02:43:02.804947 14992 raft_consensus.cc:348] T c3de2f7c6a694cfa9add33b164f548cd P cc2903be8270464099694a1d759b63f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc2903be8270464099694a1d759b63f5" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 34087 } } peers { permanent_uuid: "2079ff48d01f423b81313343b63c2be5" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 33921 } } peers { permanent_uuid: "037a824541264914af1453e1a583f168" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 41685 } }
I20260925 02:43:02.805038 14993 raft_consensus.cc:348] T c3de2f7c6a694cfa9add33b164f548cd P 037a824541264914af1453e1a583f168 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc2903be8270464099694a1d759b63f5" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 34087 } } peers { permanent_uuid: "2079ff48d01f423b81313343b63c2be5" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 33921 } } peers { permanent_uuid: "037a824541264914af1453e1a583f168" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 41685 } }
I20260925 02:43:02.805238 14993 raft_consensus.cc:374] T c3de2f7c6a694cfa9add33b164f548cd P 037a824541264914af1453e1a583f168 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:43:02.805235 14992 raft_consensus.cc:374] T c3de2f7c6a694cfa9add33b164f548cd P cc2903be8270464099694a1d759b63f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:43:02.805284 14993 raft_consensus.cc:729] T c3de2f7c6a694cfa9add33b164f548cd P 037a824541264914af1453e1a583f168 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 037a824541264914af1453e1a583f168, State: Initialized, Role: FOLLOWER
I20260925 02:43:02.805289 14992 raft_consensus.cc:729] T c3de2f7c6a694cfa9add33b164f548cd P cc2903be8270464099694a1d759b63f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc2903be8270464099694a1d759b63f5, State: Initialized, Role: FOLLOWER
I20260925 02:43:02.805783 14993 consensus_queue.cc:260] T c3de2f7c6a694cfa9add33b164f548cd P 037a824541264914af1453e1a583f168 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc2903be8270464099694a1d759b63f5" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 34087 } } peers { permanent_uuid: "2079ff48d01f423b81313343b63c2be5" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 33921 } } peers { permanent_uuid: "037a824541264914af1453e1a583f168" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 41685 } }
I20260925 02:43:02.805780 14992 consensus_queue.cc:260] T c3de2f7c6a694cfa9add33b164f548cd P cc2903be8270464099694a1d759b63f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc2903be8270464099694a1d759b63f5" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 34087 } } peers { permanent_uuid: "2079ff48d01f423b81313343b63c2be5" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 33921 } } peers { permanent_uuid: "037a824541264914af1453e1a583f168" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 41685 } }
I20260925 02:43:02.804947 14994 raft_consensus.cc:348] T c3de2f7c6a694cfa9add33b164f548cd P 2079ff48d01f423b81313343b63c2be5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc2903be8270464099694a1d759b63f5" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 34087 } } peers { permanent_uuid: "2079ff48d01f423b81313343b63c2be5" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 33921 } } peers { permanent_uuid: "037a824541264914af1453e1a583f168" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 41685 } }
I20260925 02:43:02.806398 14994 raft_consensus.cc:374] T c3de2f7c6a694cfa9add33b164f548cd P 2079ff48d01f423b81313343b63c2be5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:43:02.806459 14994 raft_consensus.cc:729] T c3de2f7c6a694cfa9add33b164f548cd P 2079ff48d01f423b81313343b63c2be5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2079ff48d01f423b81313343b63c2be5, State: Initialized, Role: FOLLOWER
I20260925 02:43:02.806753 14840 heartbeater.cc:503] Master 127.13.157.190:33727 was elected leader, sending a full tablet report...
I20260925 02:43:02.806818 14993 ts_tablet_manager.cc:1428] T c3de2f7c6a694cfa9add33b164f548cd P 037a824541264914af1453e1a583f168: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260925 02:43:02.806818 14992 ts_tablet_manager.cc:1428] T c3de2f7c6a694cfa9add33b164f548cd P cc2903be8270464099694a1d759b63f5: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260925 02:43:02.806880 14994 consensus_queue.cc:260] T c3de2f7c6a694cfa9add33b164f548cd P 2079ff48d01f423b81313343b63c2be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc2903be8270464099694a1d759b63f5" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 34087 } } peers { permanent_uuid: "2079ff48d01f423b81313343b63c2be5" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 33921 } } peers { permanent_uuid: "037a824541264914af1453e1a583f168" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 41685 } }
I20260925 02:43:02.807163 14704 heartbeater.cc:503] Master 127.13.157.190:33727 was elected leader, sending a full tablet report...
I20260925 02:43:02.807610 14975 heartbeater.cc:503] Master 127.13.157.190:33727 was elected leader, sending a full tablet report...
I20260925 02:43:02.807911 14994 ts_tablet_manager.cc:1428] T c3de2f7c6a694cfa9add33b164f548cd P 2079ff48d01f423b81313343b63c2be5: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
W20260925 02:43:02.808940 14705 tablet.cc:2369] T c3de2f7c6a694cfa9add33b164f548cd P cc2903be8270464099694a1d759b63f5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 02:43:02.861369 15001 raft_consensus.cc:482] T c3de2f7c6a694cfa9add33b164f548cd P 2079ff48d01f423b81313343b63c2be5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 02:43:02.861683 15001 raft_consensus.cc:504] T c3de2f7c6a694cfa9add33b164f548cd P 2079ff48d01f423b81313343b63c2be5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc2903be8270464099694a1d759b63f5" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 34087 } } peers { permanent_uuid: "2079ff48d01f423b81313343b63c2be5" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 33921 } } peers { permanent_uuid: "037a824541264914af1453e1a583f168" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 41685 } }
I20260925 02:43:02.863134 15001 leader_election.cc:288] T c3de2f7c6a694cfa9add33b164f548cd P 2079ff48d01f423b81313343b63c2be5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cc2903be8270464099694a1d759b63f5 (127.13.157.129:34087), 037a824541264914af1453e1a583f168 (127.13.157.130:41685)
I20260925 02:43:02.866730 14977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:02.863386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.130:41685 (local address 127.13.157.131:36345)
0925 02:43:02.863490 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:02.863505 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:02.863601 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:02.863897 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:02.863899 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:02.863924 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:43:02.864028 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:02.864034 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:02.864880 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:02.864883 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:02.865673 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:02.865682 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:02.866323 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:02.866326 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:02.866430 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:02.866453 (+ 23us) client_negotiation.cc:769] Sending connection context
0925 02:43:02.866514 (+ 61us) client_negotiation.cc:240] Negotiation successful
0925 02:43:02.866586 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260925 02:43:02.866730 14990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:02.863485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.131:36345 (local address 127.13.157.130:41685)
0925 02:43:02.863605 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:02.863610 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:02.863627 (+ 17us) server_negotiation.cc:407] Connection header received
0925 02:43:02.863681 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:02.863684 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:02.863767 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:02.863846 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:43:02.864140 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:02.864789 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:02.865792 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:02.866214 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:02.866322 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:02.866377 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:02.866538 (+ 161us) server_negotiation.cc:299] Negotiation successful
0925 02:43:02.866604 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260925 02:43:02.867465 14987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:02.863556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.131:45349 (local address 127.13.157.129:34087)
0925 02:43:02.863673 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:02.863677 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:02.864167 (+ 490us) server_negotiation.cc:407] Connection header received
0925 02:43:02.864304 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:02.864307 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:02.864350 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:02.864407 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:43:02.864634 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:02.865136 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:02.866250 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:02.866726 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:02.866843 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:02.866894 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:02.867288 (+ 394us) server_negotiation.cc:299] Negotiation successful
0925 02:43:02.867363 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260925 02:43:02.867547 15002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:02.863386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.129:34087 (local address 127.13.157.131:45349)
0925 02:43:02.864085 (+ 699us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:02.864098 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:02.864208 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:02.864429 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:02.864432 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:02.864448 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:43:02.864555 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:02.864560 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:02.865339 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:02.865343 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:02.866116 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:02.866125 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:02.867010 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:02.867016 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:02.867157 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:02.867181 (+ 24us) client_negotiation.cc:769] Sending connection context
0925 02:43:02.867253 (+ 72us) client_negotiation.cc:240] Negotiation successful
0925 02:43:02.867321 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":640,"mutex_wait_us":276,"thread_start_us":123,"threads_started":1}
I20260925 02:43:02.868343 14793 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c3de2f7c6a694cfa9add33b164f548cd" candidate_uuid: "2079ff48d01f423b81313343b63c2be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "037a824541264914af1453e1a583f168" is_pre_election: true
I20260925 02:43:02.868343 14657 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c3de2f7c6a694cfa9add33b164f548cd" candidate_uuid: "2079ff48d01f423b81313343b63c2be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc2903be8270464099694a1d759b63f5" is_pre_election: true
I20260925 02:43:02.868659 14657 raft_consensus.cc:2454] T c3de2f7c6a694cfa9add33b164f548cd P cc2903be8270464099694a1d759b63f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2079ff48d01f423b81313343b63c2be5 in term 0.
I20260925 02:43:02.868645 14793 raft_consensus.cc:2454] T c3de2f7c6a694cfa9add33b164f548cd P 037a824541264914af1453e1a583f168 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2079ff48d01f423b81313343b63c2be5 in term 0.
I20260925 02:43:02.869184 14864 leader_election.cc:302] T c3de2f7c6a694cfa9add33b164f548cd P 2079ff48d01f423b81313343b63c2be5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 037a824541264914af1453e1a583f168, 2079ff48d01f423b81313343b63c2be5, cc2903be8270464099694a1d759b63f5; no voters:
I20260925 02:43:02.869459 15001 raft_consensus.cc:2781] T c3de2f7c6a694cfa9add33b164f548cd P 2079ff48d01f423b81313343b63c2be5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 02:43:02.869539 15001 raft_consensus.cc:482] T c3de2f7c6a694cfa9add33b164f548cd P 2079ff48d01f423b81313343b63c2be5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 02:43:02.869570 15001 raft_consensus.cc:3037] T c3de2f7c6a694cfa9add33b164f548cd P 2079ff48d01f423b81313343b63c2be5 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:43:02.870592 15001 raft_consensus.cc:504] T c3de2f7c6a694cfa9add33b164f548cd P 2079ff48d01f423b81313343b63c2be5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc2903be8270464099694a1d759b63f5" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 34087 } } peers { permanent_uuid: "2079ff48d01f423b81313343b63c2be5" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 33921 } } peers { permanent_uuid: "037a824541264914af1453e1a583f168" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 41685 } }
I20260925 02:43:02.870990 15001 leader_election.cc:288] T c3de2f7c6a694cfa9add33b164f548cd P 2079ff48d01f423b81313343b63c2be5 [CANDIDATE]: Term 1 election: Requested vote from peers cc2903be8270464099694a1d759b63f5 (127.13.157.129:34087), 037a824541264914af1453e1a583f168 (127.13.157.130:41685)
I20260925 02:43:02.871340 14793 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c3de2f7c6a694cfa9add33b164f548cd" candidate_uuid: "2079ff48d01f423b81313343b63c2be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "037a824541264914af1453e1a583f168"
I20260925 02:43:02.871352 14657 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c3de2f7c6a694cfa9add33b164f548cd" candidate_uuid: "2079ff48d01f423b81313343b63c2be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc2903be8270464099694a1d759b63f5"
I20260925 02:43:02.871449 14793 raft_consensus.cc:3037] T c3de2f7c6a694cfa9add33b164f548cd P 037a824541264914af1453e1a583f168 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:43:02.871462 14657 raft_consensus.cc:3037] T c3de2f7c6a694cfa9add33b164f548cd P cc2903be8270464099694a1d759b63f5 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:43:02.872493 14657 raft_consensus.cc:2454] T c3de2f7c6a694cfa9add33b164f548cd P cc2903be8270464099694a1d759b63f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2079ff48d01f423b81313343b63c2be5 in term 1.
I20260925 02:43:02.872771 14793 raft_consensus.cc:2454] T c3de2f7c6a694cfa9add33b164f548cd P 037a824541264914af1453e1a583f168 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2079ff48d01f423b81313343b63c2be5 in term 1.
I20260925 02:43:02.872804 14864 leader_election.cc:302] T c3de2f7c6a694cfa9add33b164f548cd P 2079ff48d01f423b81313343b63c2be5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2079ff48d01f423b81313343b63c2be5, cc2903be8270464099694a1d759b63f5; no voters:
I20260925 02:43:02.873070 15001 raft_consensus.cc:2781] T c3de2f7c6a694cfa9add33b164f548cd P 2079ff48d01f423b81313343b63c2be5 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:43:02.873274 15001 raft_consensus.cc:686] T c3de2f7c6a694cfa9add33b164f548cd P 2079ff48d01f423b81313343b63c2be5 [term 1 LEADER]: Becoming Leader. State: Replica: 2079ff48d01f423b81313343b63c2be5, State: Running, Role: LEADER
I20260925 02:43:02.873610 15001 consensus_queue.cc:237] T c3de2f7c6a694cfa9add33b164f548cd P 2079ff48d01f423b81313343b63c2be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc2903be8270464099694a1d759b63f5" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 34087 } } peers { permanent_uuid: "2079ff48d01f423b81313343b63c2be5" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 33921 } } peers { permanent_uuid: "037a824541264914af1453e1a583f168" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 41685 } }
I20260925 02:43:02.876895 14512 catalog_manager.cc:5495] T c3de2f7c6a694cfa9add33b164f548cd P 2079ff48d01f423b81313343b63c2be5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2079ff48d01f423b81313343b63c2be5 (127.13.157.131). New cstate: current_term: 1 leader_uuid: "2079ff48d01f423b81313343b63c2be5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc2903be8270464099694a1d759b63f5" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 34087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2079ff48d01f423b81313343b63c2be5" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 33921 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "037a824541264914af1453e1a583f168" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 41685 } health_report { overall_health: UNKNOWN } } }
Sep 25 02:43:02 dist-test-slave-kznw krb5kdc[14467](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304181, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.157.129@KRBTEST.COM
I20260925 02:43:02.907891 14987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:02.899051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33542 (local address 127.13.157.129:34087)
0925 02:43:02.899201 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:02.899206 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:02.899471 (+ 265us) server_negotiation.cc:407] Connection header received
0925 02:43:02.899598 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:02.899601 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:02.899649 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:02.899714 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:02.900603 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:02.901104 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:02.902026 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:02.902254 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:02.902342 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:02.904821 (+ 2479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:02.904836 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:02.904851 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:43:02.904877 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:02.906310 (+ 1433us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:02.906676 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:02.906681 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:02.906683 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:02.906782 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:02.907005 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:02.907007 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:02.907008 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:02.907254 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:02.907417 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:02.907578 (+ 161us) server_negotiation.cc:299] Negotiation successful
0925 02:43:02.907681 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
Sep 25 02:43:02 dist-test-slave-kznw krb5kdc[14467](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304182, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 25 02:43:02 dist-test-slave-kznw krb5kdc[14467](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304182, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.13.157.129@KRBTEST.COM
I20260925 02:43:02.934181 14987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:02.926143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33556 (local address 127.13.157.129:34087)
0925 02:43:02.926291 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:02.926295 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:02.926354 (+ 59us) server_negotiation.cc:407] Connection header received
0925 02:43:02.926476 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:02.926478 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:02.926521 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:02.926592 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:02.927392 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:02.928017 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:02.928856 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:02.929077 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:02.929144 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:02.931465 (+ 2321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:02.931468 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:02.931470 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:43:02.931494 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:02.932659 (+ 1165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:02.933034 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:02.933036 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:02.933038 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:02.933106 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:02.933377 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:02.933379 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:02.933380 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:02.933528 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:02.933618 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:02.933915 (+ 297us) server_negotiation.cc:299] Negotiation successful
0925 02:43:02.934023 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
Sep 25 02:43:02 dist-test-slave-kznw krb5kdc[14467](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304182, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 25 02:43:02 dist-test-slave-kznw krb5kdc[14467](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304182, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.157.129@KRBTEST.COM
I20260925 02:43:02.960443 14987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:02.951610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33564 (local address 127.13.157.129:34087)
0925 02:43:02.951763 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:02.951767 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:02.951911 (+ 144us) server_negotiation.cc:407] Connection header received
0925 02:43:02.952036 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:02.952039 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:02.952088 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:02.952177 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:02.952931 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:02.953459 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:02.954385 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:02.954604 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:02.954694 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:02.957381 (+ 2687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:02.957385 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:02.957387 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:43:02.957419 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:02.958831 (+ 1412us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:02.959298 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:02.959301 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:02.959303 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:02.959379 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:02.959580 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:02.959582 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:02.959583 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:02.959742 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:02.959851 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:02.960153 (+ 302us) server_negotiation.cc:299] Negotiation successful
0925 02:43:02.960282 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
W20260925 02:43:02.961854 14617 tablet_service.cc:2908] Rejecting scan request for tablet c3de2f7c6a694cfa9add33b164f548cd: Uninitialized: clean time has not yet been initialized
I20260925 02:43:02.963815 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 14574
I20260925 02:43:02.969528 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 14709
W20260925 02:43:02.970628 14976 tablet.cc:2369] T c3de2f7c6a694cfa9add33b164f548cd P 2079ff48d01f423b81313343b63c2be5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 02:43:02.974704 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 14845
I20260925 02:43:02.979686 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 14480
2026-09-25T02:43:02Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2209 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790304178325258-13942-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 02:43:03 dist-test-slave-kznw krb5kdc[15017](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 02:43:03 dist-test-slave-kznw krb5kdc[15017](info): set up 2 sockets
Sep 25 02:43:03 dist-test-slave-kznw krb5kdc[15017](info): commencing operation
krb5kdc: starting...
W20260925 02:43:03.938028 13942 mini_kdc.cc:121] Time spent starting KDC: real 0.937s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790304178325258-13942-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790304178325258-13942-0/krb5kdc/test-user.keytab.
Sep 25 02:43:04 dist-test-slave-kznw krb5kdc[15017](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304184, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T02:43:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T02:43:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.157.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.157.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.157.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab.
Entry for principal kudu/127.13.157.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab.
I20260925 02:43:04.057650 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790304178325258-13942-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790304178325258-13942-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790304178325258-13942-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.157.190:32809
--webserver_interface=127.13.157.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab
--principal=kudu/127.13.157.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:45101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.157.190:32809
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:43:04.167526 15030 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:43:04.167788 15030 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:43:04.167827 15030 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:43:04.171805 15030 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:43:04.171859 15030 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260925 02:43:04.171875 15030 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:43:04.171891 15030 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:43:04.171905 15030 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:43:04.171921 15030 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:43:04.177327 15030 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:45101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790304178325258-13942-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790304178325258-13942-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.157.190:32809
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab
--principal=kudu/127.13.157.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.157.190:32809
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.157.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15030
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790304178325258-13942-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:43:04.178367 15030 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:43:04.179414 15030 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:43:04.187603 15035 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:43:04.187666 15030 server_base.cc:1034] running on GCE node
W20260925 02:43:04.187587 15036 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:04.187618 15038 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:43:04.188460 15030 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:43:04.189538 15030 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:43:04.190712 15030 hybrid_clock.cc:648] HybridClock initialized: now 1790304184190697 us; error 33 us; skew 500 ppm
Sep 25 02:43:04 dist-test-slave-kznw krb5kdc[15017](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304184, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:43:04.193359 15030 init.cc:377] Logged in from keytab as kudu/127.13.157.190@KRBTEST.COM (short username kudu)
I20260925 02:43:04.194209 15030 webserver.cc:460] Webserver started at http://127.13.157.190:41223/ using document root <none> and password file <none>
I20260925 02:43:04.194738 15030 fs_manager.cc:359] Metadata directory not provided
I20260925 02:43:04.194797 15030 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:43:04.195024 15030 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:43:04.197021 15030 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790304178325258-13942-0/minicluster-data/master-0/data/instance:
uuid: "ca875dfdc04b4faf9f1b8b2920cba8cf"
format_stamp: "Formatted at 2026-09-25 02:43:04 on dist-test-slave-kznw"
server_key: "06e53050028aeefc2d10d37ba9926a9a"
server_key_iv: "f7cb10af40967452bcd03c5756100255"
server_key_version: "encryptionkey@0"
I20260925 02:43:04.197552 15030 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790304178325258-13942-0/minicluster-data/master-0/wal/instance:
uuid: "ca875dfdc04b4faf9f1b8b2920cba8cf"
format_stamp: "Formatted at 2026-09-25 02:43:04 on dist-test-slave-kznw"
server_key: "06e53050028aeefc2d10d37ba9926a9a"
server_key_iv: "f7cb10af40967452bcd03c5756100255"
server_key_version: "encryptionkey@0"
I20260925 02:43:04.201308 15030 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260925 02:43:04.203748 15045 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:04.204957 15030 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260925 02:43:04.205071 15030 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790304178325258-13942-0/minicluster-data/master-0/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790304178325258-13942-0/minicluster-data/master-0/wal
uuid: "ca875dfdc04b4faf9f1b8b2920cba8cf"
format_stamp: "Formatted at 2026-09-25 02:43:04 on dist-test-slave-kznw"
server_key: "06e53050028aeefc2d10d37ba9926a9a"
server_key_iv: "f7cb10af40967452bcd03c5756100255"
server_key_version: "encryptionkey@0"
I20260925 02:43:04.205181 15030 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790304178325258-13942-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790304178325258-13942-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790304178325258-13942-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:43:04.219419 15030 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:43:04.222142 15030 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:43:04.222303 15030 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:43:04.230346 15030 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.190:32809
I20260925 02:43:04.230372 15109 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.190:32809 every 8 connection(s)
I20260925 02:43:04.231335 15030 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
I20260925 02:43:04.233633 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 15030
I20260925 02:43:04.233733 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790304178325258-13942-0/minicluster-data/master-0/wal/instance
I20260925 02:43:04.233985 13942 external_mini_cluster.cc:1428] Setting key 2ccf1a7a28a0c4d6073af95183b840b0
I20260925 02:43:04.234205 15110 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:43:04.240592 15110 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca875dfdc04b4faf9f1b8b2920cba8cf: Bootstrap starting.
Sep 25 02:43:04 dist-test-slave-kznw krb5kdc[15017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304184, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.157.190@KRBTEST.COM
I20260925 02:43:04.243093 15110 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ca875dfdc04b4faf9f1b8b2920cba8cf: Neither blocks nor log segments found. Creating new log.
I20260925 02:43:04.243801 15110 log.cc:824] T 00000000000000000000000000000000 P ca875dfdc04b4faf9f1b8b2920cba8cf: Log is configured to *not* fsync() on all Append() calls
I20260925 02:43:04.245834 15110 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca875dfdc04b4faf9f1b8b2920cba8cf: No bootstrap required, opened a new log
I20260925 02:43:04.248667 15113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:04.235411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40532 (local address 127.13.157.190:32809)
0925 02:43:04.235849 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:04.235864 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:04.235899 (+ 35us) server_negotiation.cc:407] Connection header received
0925 02:43:04.236883 (+ 984us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:04.236903 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:04.237254 (+ 351us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:04.237562 (+ 308us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:04.238389 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:04.239441 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:04.240288 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:04.240776 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:04.240930 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:04.243550 (+ 2620us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:04.243555 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:04.243568 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:43:04.243596 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:04.245784 (+ 2188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:04.246220 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:04.246225 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:04.246229 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:04.246346 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:04.246613 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:04.246615 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:04.246617 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:04.247075 (+ 458us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:04.247262 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:04.247553 (+ 291us) server_negotiation.cc:299] Negotiation successful
0925 02:43:04.247860 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":296,"thread_start_us":146,"threads_started":1}
I20260925 02:43:04.249161 15110 raft_consensus.cc:348] T 00000000000000000000000000000000 P ca875dfdc04b4faf9f1b8b2920cba8cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca875dfdc04b4faf9f1b8b2920cba8cf" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 32809 } }
I20260925 02:43:04.249385 15110 raft_consensus.cc:374] T 00000000000000000000000000000000 P ca875dfdc04b4faf9f1b8b2920cba8cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:43:04.249472 15110 raft_consensus.cc:729] T 00000000000000000000000000000000 P ca875dfdc04b4faf9f1b8b2920cba8cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca875dfdc04b4faf9f1b8b2920cba8cf, State: Initialized, Role: FOLLOWER
I20260925 02:43:04.249958 15110 consensus_queue.cc:260] T 00000000000000000000000000000000 P ca875dfdc04b4faf9f1b8b2920cba8cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca875dfdc04b4faf9f1b8b2920cba8cf" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 32809 } }
I20260925 02:43:04.250128 15110 raft_consensus.cc:388] T 00000000000000000000000000000000 P ca875dfdc04b4faf9f1b8b2920cba8cf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 02:43:04.250195 15110 raft_consensus.cc:482] T 00000000000000000000000000000000 P ca875dfdc04b4faf9f1b8b2920cba8cf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 02:43:04.250285 15110 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ca875dfdc04b4faf9f1b8b2920cba8cf [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:43:04.251276 15110 raft_consensus.cc:504] T 00000000000000000000000000000000 P ca875dfdc04b4faf9f1b8b2920cba8cf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca875dfdc04b4faf9f1b8b2920cba8cf" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 32809 } }
I20260925 02:43:04.251569 15110 leader_election.cc:302] T 00000000000000000000000000000000 P ca875dfdc04b4faf9f1b8b2920cba8cf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ca875dfdc04b4faf9f1b8b2920cba8cf; no voters:
I20260925 02:43:04.251827 15110 leader_election.cc:288] T 00000000000000000000000000000000 P ca875dfdc04b4faf9f1b8b2920cba8cf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 02:43:04.251963 15115 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ca875dfdc04b4faf9f1b8b2920cba8cf [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:43:04.252198 15115 raft_consensus.cc:686] T 00000000000000000000000000000000 P ca875dfdc04b4faf9f1b8b2920cba8cf [term 1 LEADER]: Becoming Leader. State: Replica: ca875dfdc04b4faf9f1b8b2920cba8cf, State: Running, Role: LEADER
I20260925 02:43:04.252535 15115 consensus_queue.cc:237] T 00000000000000000000000000000000 P ca875dfdc04b4faf9f1b8b2920cba8cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca875dfdc04b4faf9f1b8b2920cba8cf" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 32809 } }
I20260925 02:43:04.252925 15110 sys_catalog.cc:564] T 00000000000000000000000000000000 P ca875dfdc04b4faf9f1b8b2920cba8cf [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:43:04.254295 15116 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca875dfdc04b4faf9f1b8b2920cba8cf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ca875dfdc04b4faf9f1b8b2920cba8cf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca875dfdc04b4faf9f1b8b2920cba8cf" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 32809 } } }
I20260925 02:43:04.254338 15117 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca875dfdc04b4faf9f1b8b2920cba8cf [sys.catalog]: SysCatalogTable state changed. Reason: New leader ca875dfdc04b4faf9f1b8b2920cba8cf. Latest consensus state: current_term: 1 leader_uuid: "ca875dfdc04b4faf9f1b8b2920cba8cf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca875dfdc04b4faf9f1b8b2920cba8cf" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 32809 } } }
I20260925 02:43:04.254522 15116 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca875dfdc04b4faf9f1b8b2920cba8cf [sys.catalog]: This master's current role is: LEADER
I20260925 02:43:04.254540 15117 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca875dfdc04b4faf9f1b8b2920cba8cf [sys.catalog]: This master's current role is: LEADER
W20260925 02:43:04.257409 15130 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ca875dfdc04b4faf9f1b8b2920cba8cf: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 02:43:04.257477 15130 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 02:43:04.257575 15129 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:43:04.258610 15129 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:43:04.264222 15129 catalog_manager.cc:1347] Generated new cluster ID: 7a44ff7fa1b24d9ea011e5cc2212a912
I20260925 02:43:04.264324 15129 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 02:43:04.287819 15129 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 02:43:04.288771 15129 catalog_manager.cc:1504] Loading token signing keys...
I20260925 02:43:04.293552 15129 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ca875dfdc04b4faf9f1b8b2920cba8cf: Generated new TSK 0
I20260925 02:43:04.294224 15129 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.157.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.157.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.157.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.129.keytab.
Entry for principal kudu/127.13.157.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.129.keytab.
I20260925 02:43:04.322312 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790304178325258-13942-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790304178325258-13942-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790304178325258-13942-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790304178325258-13942-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.157.129:0
--local_ip_for_outbound_sockets=127.13.157.129
--webserver_interface=127.13.157.129
--webserver_port=0
--tserver_master_addrs=127.13.157.190:32809
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.129.keytab
--principal=kudu/127.13.157.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:45101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:43:04.446317 15136 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:43:04.446575 15136 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:43:04.446619 15136 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:43:04.450600 15136 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:43:04.450656 15136 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:43:04.450693 15136 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 02:43:04.450724 15136 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.157.129
I20260925 02:43:04.455893 15136 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:45101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790304178325258-13942-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790304178325258-13942-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.129.keytab
--principal=kudu/127.13.157.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.157.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790304178325258-13942-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.157.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.157.190:32809
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15136
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.157.129
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790304178325258-13942-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:43:04.456892 15136 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:43:04.457857 15136 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:43:04.467063 15142 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:04.467063 15144 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:43:04.467446 15136 server_base.cc:1034] running on GCE node
W20260925 02:43:04.467489 15141 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:43:04.467895 15136 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:43:04.468555 15136 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:43:04.469720 15136 hybrid_clock.cc:648] HybridClock initialized: now 1790304184469705 us; error 37 us; skew 500 ppm
Sep 25 02:43:04 dist-test-slave-kznw krb5kdc[15017](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304184, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:43:04.472321 15136 init.cc:377] Logged in from keytab as kudu/127.13.157.129@KRBTEST.COM (short username kudu)
I20260925 02:43:04.473147 15136 webserver.cc:460] Webserver started at http://127.13.157.129:35953/ using document root <none> and password file <none>
I20260925 02:43:04.473666 15136 fs_manager.cc:359] Metadata directory not provided
I20260925 02:43:04.473730 15136 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:43:04.473912 15136 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:43:04.475940 15136 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790304178325258-13942-0/minicluster-data/ts-0/data/instance:
uuid: "221d138c3adb48d6b239f506a8aaa778"
format_stamp: "Formatted at 2026-09-25 02:43:04 on dist-test-slave-kznw"
server_key: "ec21c8a324be081883e1fdbe806d9441"
server_key_iv: "0796ef553334f46a36243375f14e54b9"
server_key_version: "encryptionkey@0"
I20260925 02:43:04.476481 15136 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790304178325258-13942-0/minicluster-data/ts-0/wal/instance:
uuid: "221d138c3adb48d6b239f506a8aaa778"
format_stamp: "Formatted at 2026-09-25 02:43:04 on dist-test-slave-kznw"
server_key: "ec21c8a324be081883e1fdbe806d9441"
server_key_iv: "0796ef553334f46a36243375f14e54b9"
server_key_version: "encryptionkey@0"
I20260925 02:43:04.480088 15136 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260925 02:43:04.482398 15152 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:04.483575 15136 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260925 02:43:04.483677 15136 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790304178325258-13942-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790304178325258-13942-0/minicluster-data/ts-0/wal
uuid: "221d138c3adb48d6b239f506a8aaa778"
format_stamp: "Formatted at 2026-09-25 02:43:04 on dist-test-slave-kznw"
server_key: "ec21c8a324be081883e1fdbe806d9441"
server_key_iv: "0796ef553334f46a36243375f14e54b9"
server_key_version: "encryptionkey@0"
I20260925 02:43:04.483771 15136 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790304178325258-13942-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790304178325258-13942-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790304178325258-13942-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:43:04.502642 15136 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:43:04.505509 15136 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:43:04.505686 15136 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:43:04.507180 15136 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:43:04.507238 15136 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:04.507292 15136 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:43:04.507319 15136 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 25 02:43:04 dist-test-slave-kznw krb5kdc[15017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304184, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.129@KRBTEST.COM for kudu/127.13.157.190@KRBTEST.COM
I20260925 02:43:04.517585 15136 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.129:39953
I20260925 02:43:04.517623 15269 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.129:39953 every 8 connection(s)
I20260925 02:43:04.518523 15136 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790304178325258-13942-0/minicluster-data/ts-0/data/info.pb
I20260925 02:43:04.519155 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 15136
I20260925 02:43:04.519268 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790304178325258-13942-0/minicluster-data/ts-0/wal/instance
I20260925 02:43:04.519548 13942 external_mini_cluster.cc:1428] Setting key c60be2890e942232a9cbd794aa47be6b
I20260925 02:43:04.520406 15113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:04.507870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.129:49213 (local address 127.13.157.190:32809)
0925 02:43:04.508007 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:04.508012 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:04.508896 (+ 884us) server_negotiation.cc:407] Connection header received
0925 02:43:04.509975 (+ 1079us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:04.509979 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:04.510038 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:04.510134 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:04.511339 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:04.512240 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:04.513465 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:04.513829 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:04.513942 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:04.516704 (+ 2762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:04.516707 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:04.516709 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:43:04.516743 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:04.518199 (+ 1456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:04.518689 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:04.518693 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:04.518695 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:04.518777 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:04.519167 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:04.519170 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:04.519171 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:04.519401 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:04.519484 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:04.520117 (+ 633us) server_negotiation.cc:299] Negotiation successful
0925 02:43:04.520257 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260925 02:43:04.521286 15161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:04.508309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:32809 (local address 127.13.157.129:49213)
0925 02:43:04.508753 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:04.508804 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:04.509709 (+ 905us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:04.510345 (+ 636us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:04.510352 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:04.510668 (+ 316us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:04.511196 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:04.511209 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:04.512378 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:04.512382 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:04.513360 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:04.513371 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:04.514007 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:04.514012 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:04.514203 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:04.514750 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:04.514769 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:04.516542 (+ 1773us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:43:04.518311 (+ 1769us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:04.518316 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:04.518331 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:04.518580 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:04.518863 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:04.518867 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:04.518869 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:04.519057 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:04.519506 (+ 449us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:43:04.519513 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:43:04.519832 (+ 319us) client_negotiation.cc:769] Sending connection context
0925 02:43:04.520036 (+ 204us) client_negotiation.cc:240] Negotiation successful
0925 02:43:04.520362 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":131,"threads_started":1}
I20260925 02:43:04.523018 15270 heartbeater.cc:347] Connected to a master server at 127.13.157.190:32809
I20260925 02:43:04.523157 15270 heartbeater.cc:464] Registering TS with master...
I20260925 02:43:04.523687 15270 heartbeater.cc:511] Master 127.13.157.190:32809 requested a full tablet report, sending...
I20260925 02:43:04.525172 15061 ts_manager.cc:194] Registered new tserver with Master: 221d138c3adb48d6b239f506a8aaa778 (127.13.157.129:39953)
I20260925 02:43:04.526650 15061 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.157.129@KRBTEST.COM'} at 127.13.157.129:49213
WARNING: no policy specified for kudu/127.13.157.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.157.130@KRBTEST.COM" created.
I20260925 02:43:04.533283 15113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:04.524784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.129:56135 (local address 127.13.157.190:32809)
0925 02:43:04.524890 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:04.524894 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:04.527149 (+ 2255us) server_negotiation.cc:407] Connection header received
0925 02:43:04.527254 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:04.527258 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:04.527299 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:04.527361 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:04.527904 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:04.528394 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:04.529164 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:04.529391 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:04.529475 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:04.530495 (+ 1020us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:04.530498 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:04.530501 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:43:04.530525 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:04.531731 (+ 1206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:04.532168 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:04.532171 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:04.532173 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:04.532248 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:04.532487 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:04.532490 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:04.532492 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:04.532679 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:04.532778 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:04.533029 (+ 251us) server_negotiation.cc:299] Negotiation successful
0925 02:43:04.533133 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260925 02:43:04.533296 15277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:04.524736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:32809 (local address 127.13.157.129:56135)
0925 02:43:04.527070 (+ 2334us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:04.527082 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:04.527173 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:04.527462 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:04.527466 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:04.527664 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:04.527797 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:04.527803 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:04.528509 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:04.528512 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:04.529061 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:04.529069 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:04.529488 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:04.529491 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:04.529583 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:04.530045 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:04.530055 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:04.530377 (+ 322us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:43:04.531829 (+ 1452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:04.531832 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:04.531834 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:04.532054 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:04.532329 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:04.532332 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:04.532334 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:04.532409 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:04.532778 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:43:04.532781 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:43:04.532871 (+ 90us) client_negotiation.cc:769] Sending connection context
0925 02:43:04.532963 (+ 92us) client_negotiation.cc:240] Negotiation successful
0925 02:43:04.533099 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2261,"mutex_wait_us":10,"thread_start_us":93,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.157.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.130.keytab.
Entry for principal kudu/127.13.157.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.130.keytab.
I20260925 02:43:04.547741 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790304178325258-13942-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790304178325258-13942-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790304178325258-13942-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790304178325258-13942-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.157.130:0
--local_ip_for_outbound_sockets=127.13.157.130
--webserver_interface=127.13.157.130
--webserver_port=0
--tserver_master_addrs=127.13.157.190:32809
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.130.keytab
--principal=kudu/127.13.157.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:45101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:43:04.655503 15279 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:43:04.655802 15279 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:43:04.655867 15279 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:43:04.660077 15279 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:43:04.660162 15279 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:43:04.660219 15279 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 02:43:04.660275 15279 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.157.130
I20260925 02:43:04.665827 15279 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:45101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790304178325258-13942-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790304178325258-13942-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.130.keytab
--principal=kudu/127.13.157.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.157.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790304178325258-13942-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.157.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.157.190:32809
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15279
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.157.130
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790304178325258-13942-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:43:04.667021 15279 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:43:04.668120 15279 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:43:04.678102 15287 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:04.678056 15284 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:04.678158 15285 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:43:04.678225 15279 server_base.cc:1034] running on GCE node
I20260925 02:43:04.678814 15279 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:43:04.679526 15279 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:43:04.680709 15279 hybrid_clock.cc:648] HybridClock initialized: now 1790304184680674 us; error 37 us; skew 500 ppm
Sep 25 02:43:04 dist-test-slave-kznw krb5kdc[15017](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304184, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:43:04.683631 15279 init.cc:377] Logged in from keytab as kudu/127.13.157.130@KRBTEST.COM (short username kudu)
I20260925 02:43:04.684593 15279 webserver.cc:460] Webserver started at http://127.13.157.130:34283/ using document root <none> and password file <none>
I20260925 02:43:04.685226 15279 fs_manager.cc:359] Metadata directory not provided
I20260925 02:43:04.685297 15279 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:43:04.685513 15279 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:43:04.687733 15279 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790304178325258-13942-0/minicluster-data/ts-1/data/instance:
uuid: "7d639c741543473587167bd497e3fc7d"
format_stamp: "Formatted at 2026-09-25 02:43:04 on dist-test-slave-kznw"
server_key: "746d5937fa98b1f8184693b007f0d808"
server_key_iv: "219231509a5022c360d7143c2f6bc5ee"
server_key_version: "encryptionkey@0"
I20260925 02:43:04.688329 15279 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790304178325258-13942-0/minicluster-data/ts-1/wal/instance:
uuid: "7d639c741543473587167bd497e3fc7d"
format_stamp: "Formatted at 2026-09-25 02:43:04 on dist-test-slave-kznw"
server_key: "746d5937fa98b1f8184693b007f0d808"
server_key_iv: "219231509a5022c360d7143c2f6bc5ee"
server_key_version: "encryptionkey@0"
I20260925 02:43:04.692075 15279 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260925 02:43:04.694559 15294 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:04.695816 15279 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260925 02:43:04.695915 15279 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790304178325258-13942-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790304178325258-13942-0/minicluster-data/ts-1/wal
uuid: "7d639c741543473587167bd497e3fc7d"
format_stamp: "Formatted at 2026-09-25 02:43:04 on dist-test-slave-kznw"
server_key: "746d5937fa98b1f8184693b007f0d808"
server_key_iv: "219231509a5022c360d7143c2f6bc5ee"
server_key_version: "encryptionkey@0"
I20260925 02:43:04.696002 15279 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790304178325258-13942-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790304178325258-13942-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790304178325258-13942-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:43:04.715472 15279 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:43:04.718433 15279 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:43:04.718605 15279 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:43:04.720211 15279 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:43:04.720270 15279 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:04.720327 15279 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:43:04.720355 15279 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 25 02:43:04 dist-test-slave-kznw krb5kdc[15017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304184, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.130@KRBTEST.COM for kudu/127.13.157.190@KRBTEST.COM
I20260925 02:43:04.730684 15279 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.130:37769
I20260925 02:43:04.730742 15410 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.130:37769 every 8 connection(s)
I20260925 02:43:04.731897 15279 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790304178325258-13942-0/minicluster-data/ts-1/data/info.pb
I20260925 02:43:04.733620 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 15279
I20260925 02:43:04.733740 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790304178325258-13942-0/minicluster-data/ts-1/wal/instance
I20260925 02:43:04.734045 13942 external_mini_cluster.cc:1428] Setting key 5e47731dd0b29bd2326cb99a2ddaf222
I20260925 02:43:04.734297 15113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:04.720951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.130:35567 (local address 127.13.157.190:32809)
0925 02:43:04.721099 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:04.721104 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:04.722035 (+ 931us) server_negotiation.cc:407] Connection header received
0925 02:43:04.723179 (+ 1144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:04.723182 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:04.723244 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:04.723322 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:04.724908 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:04.725483 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:04.726718 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:04.727097 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:04.727211 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:04.730303 (+ 3092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:04.730307 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:04.730310 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:43:04.730344 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:04.732016 (+ 1672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:04.732652 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:04.732656 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:04.732658 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:04.732765 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:04.733071 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:04.733073 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:04.733074 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:04.733273 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:04.733352 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:04.733996 (+ 644us) server_negotiation.cc:299] Negotiation successful
0925 02:43:04.734140 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260925 02:43:04.735199 15303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:04.721401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:32809 (local address 127.13.157.130:35567)
0925 02:43:04.721860 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:04.721914 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:04.722929 (+ 1015us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:04.723549 (+ 620us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:04.723559 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:04.724025 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:04.724725 (+ 700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:04.724745 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:04.725605 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:04.725609 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:04.726569 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:04.726579 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:04.727270 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:04.727275 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:04.727514 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:04.728107 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:04.728130 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:04.730076 (+ 1946us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:43:04.732133 (+ 2057us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:04.732139 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:04.732164 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:04.732522 (+ 358us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:04.732845 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:04.732847 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:04.732848 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:04.732982 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:04.733352 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:43:04.733357 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:43:04.733668 (+ 311us) client_negotiation.cc:769] Sending connection context
0925 02:43:04.733919 (+ 251us) client_negotiation.cc:240] Negotiation successful
0925 02:43:04.734266 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":104,"threads_started":1}
I20260925 02:43:04.736538 15411 heartbeater.cc:347] Connected to a master server at 127.13.157.190:32809
I20260925 02:43:04.736697 15411 heartbeater.cc:464] Registering TS with master...
I20260925 02:43:04.737274 15411 heartbeater.cc:511] Master 127.13.157.190:32809 requested a full tablet report, sending...
I20260925 02:43:04.738662 15062 ts_manager.cc:194] Registered new tserver with Master: 7d639c741543473587167bd497e3fc7d (127.13.157.130:37769)
I20260925 02:43:04.739406 15062 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.157.130@KRBTEST.COM'} at 127.13.157.130:35567
WARNING: no policy specified for kudu/127.13.157.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.157.131@KRBTEST.COM" created.
I20260925 02:43:04.751422 15418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:04.739290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:32809 (local address 127.13.157.130:54407)
0925 02:43:04.744650 (+ 5360us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:04.744663 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:04.744762 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:04.744974 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:04.744976 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:04.745162 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:04.745313 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:04.745320 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:04.746071 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:04.746074 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:04.746710 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:04.746719 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:04.747204 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:04.747209 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:04.747307 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:04.747769 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:04.747785 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:04.748194 (+ 409us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:43:04.749864 (+ 1670us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:04.749867 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:04.749869 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:04.750067 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:04.750378 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:04.750380 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:04.750382 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:04.750460 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:04.750980 (+ 520us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:43:04.750983 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:43:04.751065 (+ 82us) client_negotiation.cc:769] Sending connection context
0925 02:43:04.751144 (+ 79us) client_negotiation.cc:240] Negotiation successful
0925 02:43:04.751262 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5270,"thread_start_us":109,"threads_started":1}
I20260925 02:43:04.751519 15113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:04.739334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.130:54407 (local address 127.13.157.190:32809)
0925 02:43:04.739461 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:04.739465 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:04.744747 (+ 5282us) server_negotiation.cc:407] Connection header received
0925 02:43:04.744845 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:04.744847 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:04.744890 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:04.744950 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:04.745404 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:04.745943 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:04.746824 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:04.747100 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:04.747194 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:04.748359 (+ 1165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:04.748362 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:04.748365 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:43:04.748395 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:04.749759 (+ 1364us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:04.750182 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:04.750186 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:04.750188 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:04.750288 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:04.750539 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:04.750543 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:04.750545 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:04.750854 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:04.750990 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:04.751227 (+ 237us) server_negotiation.cc:299] Negotiation successful
0925 02:43:04.751358 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.157.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.131.keytab.
Entry for principal kudu/127.13.157.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.131.keytab.
I20260925 02:43:04.762626 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790304178325258-13942-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790304178325258-13942-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790304178325258-13942-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790304178325258-13942-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.157.131:0
--local_ip_for_outbound_sockets=127.13.157.131
--webserver_interface=127.13.157.131
--webserver_port=0
--tserver_master_addrs=127.13.157.190:32809
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.131.keytab
--principal=kudu/127.13.157.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:45101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:43:04.871249 15420 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:43:04.871513 15420 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:43:04.871558 15420 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:43:04.875583 15420 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:43:04.875640 15420 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:43:04.875680 15420 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 02:43:04.875734 15420 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.157.131
I20260925 02:43:04.881068 15420 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:45101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790304178325258-13942-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790304178325258-13942-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.131.keytab
--principal=kudu/127.13.157.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.157.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790304178325258-13942-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.157.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.157.190:32809
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15420
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.157.131
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790304178325258-13942-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:43:04.882114 15420 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:43:04.883136 15420 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:43:04.892647 15429 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:04.892654 15426 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:43:04.892902 15420 server_base.cc:1034] running on GCE node
W20260925 02:43:04.892675 15427 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:43:04.893422 15420 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:43:04.894052 15420 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:43:04.895202 15420 hybrid_clock.cc:648] HybridClock initialized: now 1790304184895180 us; error 33 us; skew 500 ppm
Sep 25 02:43:04 dist-test-slave-kznw krb5kdc[15017](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304184, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:43:04.897853 15420 init.cc:377] Logged in from keytab as kudu/127.13.157.131@KRBTEST.COM (short username kudu)
I20260925 02:43:04.898756 15420 webserver.cc:460] Webserver started at http://127.13.157.131:33833/ using document root <none> and password file <none>
I20260925 02:43:04.899356 15420 fs_manager.cc:359] Metadata directory not provided
I20260925 02:43:04.899425 15420 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:43:04.899641 15420 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:43:04.901670 15420 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790304178325258-13942-0/minicluster-data/ts-2/data/instance:
uuid: "c22e9dc4838648d2bdaaa76af7ffc4b7"
format_stamp: "Formatted at 2026-09-25 02:43:04 on dist-test-slave-kznw"
server_key: "c1ed20bfa26d88f08ef7822cd8dadc9d"
server_key_iv: "085db46be67f1406e7a0d1a03081702f"
server_key_version: "encryptionkey@0"
I20260925 02:43:04.902204 15420 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790304178325258-13942-0/minicluster-data/ts-2/wal/instance:
uuid: "c22e9dc4838648d2bdaaa76af7ffc4b7"
format_stamp: "Formatted at 2026-09-25 02:43:04 on dist-test-slave-kznw"
server_key: "c1ed20bfa26d88f08ef7822cd8dadc9d"
server_key_iv: "085db46be67f1406e7a0d1a03081702f"
server_key_version: "encryptionkey@0"
I20260925 02:43:04.905807 15420 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260925 02:43:04.908373 15437 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:04.909595 15420 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260925 02:43:04.909701 15420 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790304178325258-13942-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790304178325258-13942-0/minicluster-data/ts-2/wal
uuid: "c22e9dc4838648d2bdaaa76af7ffc4b7"
format_stamp: "Formatted at 2026-09-25 02:43:04 on dist-test-slave-kznw"
server_key: "c1ed20bfa26d88f08ef7822cd8dadc9d"
server_key_iv: "085db46be67f1406e7a0d1a03081702f"
server_key_version: "encryptionkey@0"
I20260925 02:43:04.909797 15420 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790304178325258-13942-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790304178325258-13942-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790304178325258-13942-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:43:04.925451 15420 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:43:04.928650 15420 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:43:04.928859 15420 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:43:04.930354 15420 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:43:04.930403 15420 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:04.930444 15420 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:43:04.930469 15420 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:04.940575 15420 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.131:45991
Sep 25 02:43:04 dist-test-slave-kznw krb5kdc[15017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304184, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.131@KRBTEST.COM for kudu/127.13.157.190@KRBTEST.COM
I20260925 02:43:04.940637 15555 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.131:45991 every 8 connection(s)
I20260925 02:43:04.941550 15420 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790304178325258-13942-0/minicluster-data/ts-2/data/info.pb
I20260925 02:43:04.944705 15113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:04.931331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.131:39791 (local address 127.13.157.190:32809)
0925 02:43:04.931449 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:04.931453 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:04.932567 (+ 1114us) server_negotiation.cc:407] Connection header received
0925 02:43:04.933832 (+ 1265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:04.933836 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:04.933903 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:04.933984 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:04.935507 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:04.936331 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:04.937997 (+ 1666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:04.938343 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:04.938460 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:04.941276 (+ 2816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:04.941281 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:04.941283 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:43:04.941313 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:04.942783 (+ 1470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:04.943278 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:04.943281 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:04.943283 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:04.943360 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:04.943673 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:04.943676 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:04.943678 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:04.943859 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:04.943946 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:04.944421 (+ 475us) server_negotiation.cc:299] Negotiation successful
0925 02:43:04.944542 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260925 02:43:04.945467 15446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:04.931894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:32809 (local address 127.13.157.131:39791)
0925 02:43:04.932416 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:04.932462 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:04.933580 (+ 1118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:04.934284 (+ 704us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:04.934291 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:04.934663 (+ 372us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:04.935351 (+ 688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:04.935368 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:04.936465 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:04.936469 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:04.937874 (+ 1405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:04.937884 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:04.938505 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:04.938508 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:04.938672 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:04.939344 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:04.939365 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:04.941088 (+ 1723us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:43:04.942889 (+ 1801us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:04.942895 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:04.942930 (+ 35us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:04.943180 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:04.943443 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:04.943445 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:04.943448 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:04.943579 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:04.943956 (+ 377us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:43:04.943963 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:43:04.944192 (+ 229us) client_negotiation.cc:769] Sending connection context
0925 02:43:04.944347 (+ 155us) client_negotiation.cc:240] Negotiation successful
0925 02:43:04.944656 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":155,"threads_started":1}
I20260925 02:43:04.946564 15556 heartbeater.cc:347] Connected to a master server at 127.13.157.190:32809
I20260925 02:43:04.946729 15556 heartbeater.cc:464] Registering TS with master...
I20260925 02:43:04.947293 15556 heartbeater.cc:511] Master 127.13.157.190:32809 requested a full tablet report, sending...
I20260925 02:43:04.948387 15061 ts_manager.cc:194] Registered new tserver with Master: c22e9dc4838648d2bdaaa76af7ffc4b7 (127.13.157.131:45991)
I20260925 02:43:04.948724 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 15420
I20260925 02:43:04.948840 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790304178325258-13942-0/minicluster-data/ts-2/wal/instance
I20260925 02:43:04.949090 15061 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.157.131@KRBTEST.COM'} at 127.13.157.131:39791
I20260925 02:43:04.949119 13942 external_mini_cluster.cc:1428] Setting key ebc70a958847a2daa4dda806f2f0f6b7
I20260925 02:43:04.951840 13942 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 02:43:04.956233 15562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:04.948878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:32809 (local address 127.13.157.131:44981)
0925 02:43:04.949327 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:04.949342 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:04.949443 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:04.949723 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:04.949726 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:04.949938 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:04.950045 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:04.950053 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:04.950787 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:04.950790 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:04.951555 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:04.951566 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:04.952275 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:04.952279 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:04.952360 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:04.952830 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:04.952840 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:04.953142 (+ 302us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:43:04.954847 (+ 1705us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:04.954849 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:04.954851 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:04.955104 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:04.955379 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:04.955381 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:04.955383 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:04.955461 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:04.955812 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:43:04.955814 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:43:04.955892 (+ 78us) client_negotiation.cc:769] Sending connection context
0925 02:43:04.955969 (+ 77us) client_negotiation.cc:240] Negotiation successful
0925 02:43:04.956080 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":105,"threads_started":1}
I20260925 02:43:04.956314 15113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:04.949223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.131:44981 (local address 127.13.157.190:32809)
0925 02:43:04.949344 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:04.949348 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:04.949397 (+ 49us) server_negotiation.cc:407] Connection header received
0925 02:43:04.949565 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:04.949569 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:04.949617 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:04.949694 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:04.950164 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:04.950664 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:04.951678 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:04.952034 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:04.952139 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:04.953262 (+ 1123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:04.953265 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:04.953268 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:43:04.953292 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:04.954723 (+ 1431us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:04.955204 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:04.955208 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:04.955209 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:04.955293 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:04.955529 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:04.955532 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:04.955533 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:04.955725 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:04.955823 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:04.956037 (+ 214us) server_negotiation.cc:299] Negotiation successful
0925 02:43:04.956151 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
Sep 25 02:43:04 dist-test-slave-kznw krb5kdc[15017](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304184, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 25 02:43:04 dist-test-slave-kznw krb5kdc[15017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304184, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.13.157.190@KRBTEST.COM
I20260925 02:43:04.976490 15113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:04.969253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40536 (local address 127.13.157.190:32809)
0925 02:43:04.969369 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:04.969373 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:04.969385 (+ 12us) server_negotiation.cc:407] Connection header received
0925 02:43:04.969423 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:04.969425 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:04.969463 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:04.969528 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:04.970169 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:04.970670 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:04.971536 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:04.971758 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:04.971830 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:04.973892 (+ 2062us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:04.973895 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:04.973898 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:43:04.973920 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:04.975201 (+ 1281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:04.975539 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:04.975542 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:04.975543 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:04.975606 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:04.975789 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:04.975791 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:04.975792 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:04.975946 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:04.976041 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:04.976275 (+ 234us) server_negotiation.cc:299] Negotiation successful
0925 02:43:04.976370 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260925 02:43:04.979220 15061 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40536:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 02:43:04.982252 15061 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 02:43:04.996275 15574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:04.990388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.129:39953 (local address 127.0.0.1:38420)
0925 02:43:04.991539 (+ 1151us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:04.991550 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:04.991615 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:04.992507 (+ 892us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:04.992510 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:04.992528 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:43:04.992621 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:04.992627 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:04.993913 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:04.993917 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:04.995033 (+ 1116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:04.995041 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:04.995927 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:04.995930 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:04.996021 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:04.996041 (+ 20us) client_negotiation.cc:769] Sending connection context
0925 02:43:04.996085 (+ 44us) client_negotiation.cc:240] Negotiation successful
0925 02:43:04.996155 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1092,"thread_start_us":87,"threads_started":1}
I20260925 02:43:04.996141 15578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:04.991058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.131:45991 (local address 127.0.0.1:33686)
0925 02:43:04.991792 (+ 734us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:04.991805 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:04.991890 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:04.992302 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:04.992305 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:04.992322 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:43:04.992436 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:04.992441 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:04.993450 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:04.993453 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:04.994592 (+ 1139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:04.994601 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:04.995593 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:04.995597 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:04.995724 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:04.995769 (+ 45us) client_negotiation.cc:769] Sending connection context
0925 02:43:04.995873 (+ 104us) client_negotiation.cc:240] Negotiation successful
0925 02:43:04.995988 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":652,"mutex_wait_us":1,"thread_start_us":80,"threads_started":1}
I20260925 02:43:04.996445 15576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:04.990440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50988 (local address 127.13.157.130:37769)
0925 02:43:04.990749 (+ 309us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:04.990763 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:04.991291 (+ 528us) server_negotiation.cc:407] Connection header received
0925 02:43:04.991477 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:04.991482 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:04.991701 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:04.992215 (+ 514us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:43:04.992343 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:04.993486 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:04.994967 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:04.995746 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:04.995872 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:04.995996 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:04.996230 (+ 234us) server_negotiation.cc:299] Negotiation successful
0925 02:43:04.996317 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":210,"thread_start_us":101,"threads_started":1}
I20260925 02:43:04.996279 15575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:04.990301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.130:37769 (local address 127.0.0.1:50988)
0925 02:43:04.991145 (+ 844us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:04.991193 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:04.991326 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:04.991873 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:04.991879 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:04.991905 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:43:04.992041 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:04.992048 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:04.993613 (+ 1565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:04.993617 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:04.994827 (+ 1210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:04.994837 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:04.995856 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:04.995860 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:04.995945 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:04.995967 (+ 22us) client_negotiation.cc:769] Sending connection context
0925 02:43:04.996052 (+ 85us) client_negotiation.cc:240] Negotiation successful
0925 02:43:04.996124 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":666,"thread_start_us":71,"threads_started":1}
I20260925 02:43:04.996328 15579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:04.991517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33686 (local address 127.13.157.131:45991)
0925 02:43:04.991868 (+ 351us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:04.991879 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:04.991900 (+ 21us) server_negotiation.cc:407] Connection header received
0925 02:43:04.992035 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:04.992039 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:04.992177 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:04.992312 (+ 135us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:43:04.992552 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:04.993348 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:04.994705 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:04.995463 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:04.995592 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:04.995720 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:04.996087 (+ 367us) server_negotiation.cc:299] Negotiation successful
0925 02:43:04.996178 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":252,"thread_start_us":103,"threads_started":1}
I20260925 02:43:04.997118 15577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:04.990936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38420 (local address 127.13.157.129:39953)
0925 02:43:04.992087 (+ 1151us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:04.992103 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:04.992124 (+ 21us) server_negotiation.cc:407] Connection header received
0925 02:43:04.992198 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:04.992203 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:04.992379 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:04.992518 (+ 139us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:43:04.992713 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:04.993797 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:04.995135 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:04.995796 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:04.996754 (+ 958us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:04.996848 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:04.996897 (+ 49us) server_negotiation.cc:299] Negotiation successful
0925 02:43:04.996983 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1050,"thread_start_us":79,"threads_started":1}
I20260925 02:43:04.999377 15488 tablet_service.cc:1467] Processing CreateTablet for tablet 95ca5e1f18a443c2a69c7ca9affac919 (DEFAULT_TABLE table=test-table [id=aea6c7844041492b9ebc813f14bc3201]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:43:04.999521 15344 tablet_service.cc:1467] Processing CreateTablet for tablet 95ca5e1f18a443c2a69c7ca9affac919 (DEFAULT_TABLE table=test-table [id=aea6c7844041492b9ebc813f14bc3201]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:43:04.999521 15202 tablet_service.cc:1467] Processing CreateTablet for tablet 95ca5e1f18a443c2a69c7ca9affac919 (DEFAULT_TABLE table=test-table [id=aea6c7844041492b9ebc813f14bc3201]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:43:05.000412 15488 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95ca5e1f18a443c2a69c7ca9affac919. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:43:05.000545 15202 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95ca5e1f18a443c2a69c7ca9affac919. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:43:05.000545 15344 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95ca5e1f18a443c2a69c7ca9affac919. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:43:05.007050 15580 tablet_bootstrap.cc:492] T 95ca5e1f18a443c2a69c7ca9affac919 P c22e9dc4838648d2bdaaa76af7ffc4b7: Bootstrap starting.
I20260925 02:43:05.008451 15581 tablet_bootstrap.cc:492] T 95ca5e1f18a443c2a69c7ca9affac919 P 221d138c3adb48d6b239f506a8aaa778: Bootstrap starting.
I20260925 02:43:05.008831 15582 tablet_bootstrap.cc:492] T 95ca5e1f18a443c2a69c7ca9affac919 P 7d639c741543473587167bd497e3fc7d: Bootstrap starting.
I20260925 02:43:05.009547 15580 tablet_bootstrap.cc:654] T 95ca5e1f18a443c2a69c7ca9affac919 P c22e9dc4838648d2bdaaa76af7ffc4b7: Neither blocks nor log segments found. Creating new log.
I20260925 02:43:05.010398 15580 log.cc:824] T 95ca5e1f18a443c2a69c7ca9affac919 P c22e9dc4838648d2bdaaa76af7ffc4b7: Log is configured to *not* fsync() on all Append() calls
I20260925 02:43:05.010574 15582 tablet_bootstrap.cc:654] T 95ca5e1f18a443c2a69c7ca9affac919 P 7d639c741543473587167bd497e3fc7d: Neither blocks nor log segments found. Creating new log.
I20260925 02:43:05.010674 15581 tablet_bootstrap.cc:654] T 95ca5e1f18a443c2a69c7ca9affac919 P 221d138c3adb48d6b239f506a8aaa778: Neither blocks nor log segments found. Creating new log.
I20260925 02:43:05.011504 15581 log.cc:824] T 95ca5e1f18a443c2a69c7ca9affac919 P 221d138c3adb48d6b239f506a8aaa778: Log is configured to *not* fsync() on all Append() calls
I20260925 02:43:05.011521 15582 log.cc:824] T 95ca5e1f18a443c2a69c7ca9affac919 P 7d639c741543473587167bd497e3fc7d: Log is configured to *not* fsync() on all Append() calls
I20260925 02:43:05.012624 15580 tablet_bootstrap.cc:492] T 95ca5e1f18a443c2a69c7ca9affac919 P c22e9dc4838648d2bdaaa76af7ffc4b7: No bootstrap required, opened a new log
I20260925 02:43:05.012815 15580 ts_tablet_manager.cc:1397] T 95ca5e1f18a443c2a69c7ca9affac919 P c22e9dc4838648d2bdaaa76af7ffc4b7: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260925 02:43:05.013355 15582 tablet_bootstrap.cc:492] T 95ca5e1f18a443c2a69c7ca9affac919 P 7d639c741543473587167bd497e3fc7d: No bootstrap required, opened a new log
I20260925 02:43:05.013528 15582 ts_tablet_manager.cc:1397] T 95ca5e1f18a443c2a69c7ca9affac919 P 7d639c741543473587167bd497e3fc7d: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260925 02:43:05.013543 15581 tablet_bootstrap.cc:492] T 95ca5e1f18a443c2a69c7ca9affac919 P 221d138c3adb48d6b239f506a8aaa778: No bootstrap required, opened a new log
I20260925 02:43:05.013720 15581 ts_tablet_manager.cc:1397] T 95ca5e1f18a443c2a69c7ca9affac919 P 221d138c3adb48d6b239f506a8aaa778: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260925 02:43:05.016433 15582 raft_consensus.cc:348] T 95ca5e1f18a443c2a69c7ca9affac919 P 7d639c741543473587167bd497e3fc7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d639c741543473587167bd497e3fc7d" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 37769 } } peers { permanent_uuid: "221d138c3adb48d6b239f506a8aaa778" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 39953 } } peers { permanent_uuid: "c22e9dc4838648d2bdaaa76af7ffc4b7" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 45991 } }
I20260925 02:43:05.016657 15582 raft_consensus.cc:374] T 95ca5e1f18a443c2a69c7ca9affac919 P 7d639c741543473587167bd497e3fc7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:43:05.016718 15582 raft_consensus.cc:729] T 95ca5e1f18a443c2a69c7ca9affac919 P 7d639c741543473587167bd497e3fc7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d639c741543473587167bd497e3fc7d, State: Initialized, Role: FOLLOWER
I20260925 02:43:05.016727 15580 raft_consensus.cc:348] T 95ca5e1f18a443c2a69c7ca9affac919 P c22e9dc4838648d2bdaaa76af7ffc4b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d639c741543473587167bd497e3fc7d" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 37769 } } peers { permanent_uuid: "221d138c3adb48d6b239f506a8aaa778" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 39953 } } peers { permanent_uuid: "c22e9dc4838648d2bdaaa76af7ffc4b7" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 45991 } }
I20260925 02:43:05.016978 15580 raft_consensus.cc:374] T 95ca5e1f18a443c2a69c7ca9affac919 P c22e9dc4838648d2bdaaa76af7ffc4b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:43:05.017041 15580 raft_consensus.cc:729] T 95ca5e1f18a443c2a69c7ca9affac919 P c22e9dc4838648d2bdaaa76af7ffc4b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c22e9dc4838648d2bdaaa76af7ffc4b7, State: Initialized, Role: FOLLOWER
I20260925 02:43:05.017163 15582 consensus_queue.cc:260] T 95ca5e1f18a443c2a69c7ca9affac919 P 7d639c741543473587167bd497e3fc7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d639c741543473587167bd497e3fc7d" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 37769 } } peers { permanent_uuid: "221d138c3adb48d6b239f506a8aaa778" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 39953 } } peers { permanent_uuid: "c22e9dc4838648d2bdaaa76af7ffc4b7" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 45991 } }
I20260925 02:43:05.017473 15580 consensus_queue.cc:260] T 95ca5e1f18a443c2a69c7ca9affac919 P c22e9dc4838648d2bdaaa76af7ffc4b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d639c741543473587167bd497e3fc7d" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 37769 } } peers { permanent_uuid: "221d138c3adb48d6b239f506a8aaa778" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 39953 } } peers { permanent_uuid: "c22e9dc4838648d2bdaaa76af7ffc4b7" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 45991 } }
I20260925 02:43:05.017814 15581 raft_consensus.cc:348] T 95ca5e1f18a443c2a69c7ca9affac919 P 221d138c3adb48d6b239f506a8aaa778 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d639c741543473587167bd497e3fc7d" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 37769 } } peers { permanent_uuid: "221d138c3adb48d6b239f506a8aaa778" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 39953 } } peers { permanent_uuid: "c22e9dc4838648d2bdaaa76af7ffc4b7" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 45991 } }
I20260925 02:43:05.017987 15411 heartbeater.cc:503] Master 127.13.157.190:32809 was elected leader, sending a full tablet report...
I20260925 02:43:05.018051 15581 raft_consensus.cc:374] T 95ca5e1f18a443c2a69c7ca9affac919 P 221d138c3adb48d6b239f506a8aaa778 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:43:05.018095 15581 raft_consensus.cc:729] T 95ca5e1f18a443c2a69c7ca9affac919 P 221d138c3adb48d6b239f506a8aaa778 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 221d138c3adb48d6b239f506a8aaa778, State: Initialized, Role: FOLLOWER
I20260925 02:43:05.018553 15580 ts_tablet_manager.cc:1428] T 95ca5e1f18a443c2a69c7ca9affac919 P c22e9dc4838648d2bdaaa76af7ffc4b7: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260925 02:43:05.018524 15581 consensus_queue.cc:260] T 95ca5e1f18a443c2a69c7ca9affac919 P 221d138c3adb48d6b239f506a8aaa778 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d639c741543473587167bd497e3fc7d" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 37769 } } peers { permanent_uuid: "221d138c3adb48d6b239f506a8aaa778" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 39953 } } peers { permanent_uuid: "c22e9dc4838648d2bdaaa76af7ffc4b7" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 45991 } }
I20260925 02:43:05.018884 15556 heartbeater.cc:503] Master 127.13.157.190:32809 was elected leader, sending a full tablet report...
I20260925 02:43:05.019248 15582 ts_tablet_manager.cc:1428] T 95ca5e1f18a443c2a69c7ca9affac919 P 7d639c741543473587167bd497e3fc7d: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260925 02:43:05.019456 15581 ts_tablet_manager.cc:1428] T 95ca5e1f18a443c2a69c7ca9affac919 P 221d138c3adb48d6b239f506a8aaa778: Time spent starting tablet: real 0.006s user 0.002s sys 0.005s
W20260925 02:43:05.019721 15271 tablet.cc:2369] T 95ca5e1f18a443c2a69c7ca9affac919 P 221d138c3adb48d6b239f506a8aaa778: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 02:43:05.019809 15270 heartbeater.cc:503] Master 127.13.157.190:32809 was elected leader, sending a full tablet report...
I20260925 02:43:05.100780 15588 raft_consensus.cc:482] T 95ca5e1f18a443c2a69c7ca9affac919 P c22e9dc4838648d2bdaaa76af7ffc4b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 02:43:05.100993 15588 raft_consensus.cc:504] T 95ca5e1f18a443c2a69c7ca9affac919 P c22e9dc4838648d2bdaaa76af7ffc4b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d639c741543473587167bd497e3fc7d" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 37769 } } peers { permanent_uuid: "221d138c3adb48d6b239f506a8aaa778" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 39953 } } peers { permanent_uuid: "c22e9dc4838648d2bdaaa76af7ffc4b7" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 45991 } }
I20260925 02:43:05.102159 15588 leader_election.cc:288] T 95ca5e1f18a443c2a69c7ca9affac919 P c22e9dc4838648d2bdaaa76af7ffc4b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7d639c741543473587167bd497e3fc7d (127.13.157.130:37769), 221d138c3adb48d6b239f506a8aaa778 (127.13.157.129:39953)
I20260925 02:43:05.106786 15577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:05.102565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.131:42267 (local address 127.13.157.129:39953)
0925 02:43:05.102694 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:05.102698 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:05.102715 (+ 17us) server_negotiation.cc:407] Connection header received
0925 02:43:05.102867 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:05.102869 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:05.102951 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:05.103030 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:43:05.103303 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:05.104083 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:05.105679 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:05.106215 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:05.106351 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:05.106407 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:05.106582 (+ 175us) server_negotiation.cc:299] Negotiation successful
0925 02:43:05.106667 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260925 02:43:05.106801 15446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:05.102468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.129:39953 (local address 127.13.157.131:42267)
0925 02:43:05.102649 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:05.102662 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:05.102750 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:05.103049 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:05.103052 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:05.103066 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:43:05.103188 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:05.103194 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:05.104233 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:05.104236 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:05.105537 (+ 1301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:05.105550 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:05.106351 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:05.106354 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:05.106451 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:05.106474 (+ 23us) client_negotiation.cc:769] Sending connection context
0925 02:43:05.106546 (+ 72us) client_negotiation.cc:240] Negotiation successful
0925 02:43:05.106629 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117}
I20260925 02:43:05.106986 15590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:05.102506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.130:37769 (local address 127.13.157.131:48423)
0925 02:43:05.102927 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:05.102939 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:05.103033 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:05.103246 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:05.103249 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:05.103267 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:43:05.103364 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:05.103368 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:05.104264 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:05.104266 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:05.105791 (+ 1525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:05.105800 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:05.106559 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:05.106562 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:05.106682 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:05.106704 (+ 22us) client_negotiation.cc:769] Sending connection context
0925 02:43:05.106774 (+ 70us) client_negotiation.cc:240] Negotiation successful
0925 02:43:05.106839 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":340,"mutex_wait_us":1,"thread_start_us":166,"threads_started":1}
I20260925 02:43:05.107014 15576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:05.102489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.131:48423 (local address 127.13.157.130:37769)
0925 02:43:05.102615 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:05.102619 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:05.103019 (+ 400us) server_negotiation.cc:407] Connection header received
0925 02:43:05.103117 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:05.103120 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:05.103164 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:05.103221 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:43:05.103453 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:05.104179 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:05.105908 (+ 1729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:05.106450 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:05.106548 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:05.106596 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:05.106801 (+ 205us) server_negotiation.cc:299] Negotiation successful
0925 02:43:05.106878 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260925 02:43:05.107570 15223 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95ca5e1f18a443c2a69c7ca9affac919" candidate_uuid: "c22e9dc4838648d2bdaaa76af7ffc4b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "221d138c3adb48d6b239f506a8aaa778" is_pre_election: true
I20260925 02:43:05.107630 15364 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95ca5e1f18a443c2a69c7ca9affac919" candidate_uuid: "c22e9dc4838648d2bdaaa76af7ffc4b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d639c741543473587167bd497e3fc7d" is_pre_election: true
I20260925 02:43:05.107946 15364 raft_consensus.cc:2454] T 95ca5e1f18a443c2a69c7ca9affac919 P 7d639c741543473587167bd497e3fc7d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c22e9dc4838648d2bdaaa76af7ffc4b7 in term 0.
I20260925 02:43:05.107947 15223 raft_consensus.cc:2454] T 95ca5e1f18a443c2a69c7ca9affac919 P 221d138c3adb48d6b239f506a8aaa778 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c22e9dc4838648d2bdaaa76af7ffc4b7 in term 0.
I20260925 02:43:05.108493 15438 leader_election.cc:302] T 95ca5e1f18a443c2a69c7ca9affac919 P c22e9dc4838648d2bdaaa76af7ffc4b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 221d138c3adb48d6b239f506a8aaa778, c22e9dc4838648d2bdaaa76af7ffc4b7; no voters:
I20260925 02:43:05.108788 15588 raft_consensus.cc:2781] T 95ca5e1f18a443c2a69c7ca9affac919 P c22e9dc4838648d2bdaaa76af7ffc4b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 02:43:05.108911 15588 raft_consensus.cc:482] T 95ca5e1f18a443c2a69c7ca9affac919 P c22e9dc4838648d2bdaaa76af7ffc4b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 02:43:05.108959 15588 raft_consensus.cc:3037] T 95ca5e1f18a443c2a69c7ca9affac919 P c22e9dc4838648d2bdaaa76af7ffc4b7 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:43:05.110195 15588 raft_consensus.cc:504] T 95ca5e1f18a443c2a69c7ca9affac919 P c22e9dc4838648d2bdaaa76af7ffc4b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d639c741543473587167bd497e3fc7d" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 37769 } } peers { permanent_uuid: "221d138c3adb48d6b239f506a8aaa778" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 39953 } } peers { permanent_uuid: "c22e9dc4838648d2bdaaa76af7ffc4b7" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 45991 } }
I20260925 02:43:05.110538 15588 leader_election.cc:288] T 95ca5e1f18a443c2a69c7ca9affac919 P c22e9dc4838648d2bdaaa76af7ffc4b7 [CANDIDATE]: Term 1 election: Requested vote from peers 7d639c741543473587167bd497e3fc7d (127.13.157.130:37769), 221d138c3adb48d6b239f506a8aaa778 (127.13.157.129:39953)
I20260925 02:43:05.110885 15223 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95ca5e1f18a443c2a69c7ca9affac919" candidate_uuid: "c22e9dc4838648d2bdaaa76af7ffc4b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "221d138c3adb48d6b239f506a8aaa778"
I20260925 02:43:05.110885 15364 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95ca5e1f18a443c2a69c7ca9affac919" candidate_uuid: "c22e9dc4838648d2bdaaa76af7ffc4b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d639c741543473587167bd497e3fc7d"
I20260925 02:43:05.111043 15223 raft_consensus.cc:3037] T 95ca5e1f18a443c2a69c7ca9affac919 P 221d138c3adb48d6b239f506a8aaa778 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:43:05.111043 15364 raft_consensus.cc:3037] T 95ca5e1f18a443c2a69c7ca9affac919 P 7d639c741543473587167bd497e3fc7d [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:43:05.112287 15223 raft_consensus.cc:2454] T 95ca5e1f18a443c2a69c7ca9affac919 P 221d138c3adb48d6b239f506a8aaa778 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c22e9dc4838648d2bdaaa76af7ffc4b7 in term 1.
I20260925 02:43:05.112564 15438 leader_election.cc:302] T 95ca5e1f18a443c2a69c7ca9affac919 P c22e9dc4838648d2bdaaa76af7ffc4b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 221d138c3adb48d6b239f506a8aaa778, c22e9dc4838648d2bdaaa76af7ffc4b7; no voters:
I20260925 02:43:05.112710 15588 raft_consensus.cc:2781] T 95ca5e1f18a443c2a69c7ca9affac919 P c22e9dc4838648d2bdaaa76af7ffc4b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:43:05.112942 15588 raft_consensus.cc:686] T 95ca5e1f18a443c2a69c7ca9affac919 P c22e9dc4838648d2bdaaa76af7ffc4b7 [term 1 LEADER]: Becoming Leader. State: Replica: c22e9dc4838648d2bdaaa76af7ffc4b7, State: Running, Role: LEADER
I20260925 02:43:05.113302 15588 consensus_queue.cc:237] T 95ca5e1f18a443c2a69c7ca9affac919 P c22e9dc4838648d2bdaaa76af7ffc4b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d639c741543473587167bd497e3fc7d" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 37769 } } peers { permanent_uuid: "221d138c3adb48d6b239f506a8aaa778" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 39953 } } peers { permanent_uuid: "c22e9dc4838648d2bdaaa76af7ffc4b7" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 45991 } }
I20260925 02:43:05.113710 15364 raft_consensus.cc:2454] T 95ca5e1f18a443c2a69c7ca9affac919 P 7d639c741543473587167bd497e3fc7d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c22e9dc4838648d2bdaaa76af7ffc4b7 in term 1.
I20260925 02:43:05.116562 15060 catalog_manager.cc:5495] T 95ca5e1f18a443c2a69c7ca9affac919 P c22e9dc4838648d2bdaaa76af7ffc4b7 reported cstate change: term changed from 0 to 1, leader changed from <none> to c22e9dc4838648d2bdaaa76af7ffc4b7 (127.13.157.131). New cstate: current_term: 1 leader_uuid: "c22e9dc4838648d2bdaaa76af7ffc4b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d639c741543473587167bd497e3fc7d" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 37769 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "221d138c3adb48d6b239f506a8aaa778" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 39953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c22e9dc4838648d2bdaaa76af7ffc4b7" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 45991 } health_report { overall_health: HEALTHY } } }
I20260925 02:43:05.152913 15579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:05.149158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33690 (local address 127.13.157.131:45991)
0925 02:43:05.149300 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:05.149305 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:05.149321 (+ 16us) server_negotiation.cc:407] Connection header received
0925 02:43:05.149501 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:05.149504 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:05.149560 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:05.149643 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 02:43:05.149935 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:05.150440 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:05.151374 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:05.151590 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:05.151686 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:05.152012 (+ 326us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 02:43:05.152484 (+ 472us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 02:43:05.152590 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:05.152719 (+ 129us) server_negotiation.cc:299] Negotiation successful
0925 02:43:05.152776 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260925 02:43:05.158847 15364 raft_consensus.cc:1261] T 95ca5e1f18a443c2a69c7ca9affac919 P 7d639c741543473587167bd497e3fc7d [term 1 FOLLOWER]: Refusing update from remote peer c22e9dc4838648d2bdaaa76af7ffc4b7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:43:05.158847 15223 raft_consensus.cc:1261] T 95ca5e1f18a443c2a69c7ca9affac919 P 221d138c3adb48d6b239f506a8aaa778 [term 1 FOLLOWER]: Refusing update from remote peer c22e9dc4838648d2bdaaa76af7ffc4b7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:43:05.159626 15591 consensus_queue.cc:1035] T 95ca5e1f18a443c2a69c7ca9affac919 P c22e9dc4838648d2bdaaa76af7ffc4b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "221d138c3adb48d6b239f506a8aaa778" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 39953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:43:05.159823 15588 consensus_queue.cc:1035] T 95ca5e1f18a443c2a69c7ca9affac919 P c22e9dc4838648d2bdaaa76af7ffc4b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d639c741543473587167bd497e3fc7d" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 37769 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:43:05.168133 15596 mvcc.cc:205] Tried to move back new op lower bound from 7333085942403338240 to 7333085942227464192. Current Snapshot: MvccSnapshot[applied={T|T < 7333085942403338240}]
I20260925 02:43:05.169088 15595 mvcc.cc:205] Tried to move back new op lower bound from 7333085942403338240 to 7333085942227464192. Current Snapshot: MvccSnapshot[applied={T|T < 7333085942403338240}]
I20260925 02:43:05.180328 15062 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40536:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 02:43:05.180501 15062 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40536:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 02:43:05.182940 15062 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ca875dfdc04b4faf9f1b8b2920cba8cf: Sending DeleteTablet for 3 replicas of tablet 95ca5e1f18a443c2a69c7ca9affac919
I20260925 02:43:05.183766 15202 tablet_service.cc:1514] Processing DeleteTablet for tablet 95ca5e1f18a443c2a69c7ca9affac919 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:43:05 UTC) from {username='kudu', principal='kudu/127.13.157.190@KRBTEST.COM'} at 127.0.0.1:38420
I20260925 02:43:05.183897 15344 tablet_service.cc:1514] Processing DeleteTablet for tablet 95ca5e1f18a443c2a69c7ca9affac919 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:43:05 UTC) from {username='kudu', principal='kudu/127.13.157.190@KRBTEST.COM'} at 127.0.0.1:50988
I20260925 02:43:05.184368 15604 tablet_replica.cc:330] T 95ca5e1f18a443c2a69c7ca9affac919 P 221d138c3adb48d6b239f506a8aaa778: stopping tablet replica
I20260925 02:43:05.184485 15062 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40536:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 02:43:05.184693 15604 raft_consensus.cc:2229] T 95ca5e1f18a443c2a69c7ca9affac919 P 221d138c3adb48d6b239f506a8aaa778 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260925 02:43:05.184897 15062 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 02:43:05.184926 15605 tablet_replica.cc:330] T 95ca5e1f18a443c2a69c7ca9affac919 P 7d639c741543473587167bd497e3fc7d: stopping tablet replica
I20260925 02:43:05.185009 15604 raft_consensus.cc:2258] T 95ca5e1f18a443c2a69c7ca9affac919 P 221d138c3adb48d6b239f506a8aaa778 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:43:05.185345 15605 raft_consensus.cc:2229] T 95ca5e1f18a443c2a69c7ca9affac919 P 7d639c741543473587167bd497e3fc7d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 02:43:05.185667 15605 raft_consensus.cc:2258] T 95ca5e1f18a443c2a69c7ca9affac919 P 7d639c741543473587167bd497e3fc7d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:43:05.185681 15488 tablet_service.cc:1514] Processing DeleteTablet for tablet 95ca5e1f18a443c2a69c7ca9affac919 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:43:05 UTC) from {username='kudu', principal='kudu/127.13.157.190@KRBTEST.COM'} at 127.0.0.1:33686
I20260925 02:43:05.186129 15606 tablet_replica.cc:330] T 95ca5e1f18a443c2a69c7ca9affac919 P c22e9dc4838648d2bdaaa76af7ffc4b7: stopping tablet replica
I20260925 02:43:05.186443 15606 raft_consensus.cc:2229] T 95ca5e1f18a443c2a69c7ca9affac919 P c22e9dc4838648d2bdaaa76af7ffc4b7 [term 1 LEADER]: Raft consensus shutting down.
I20260925 02:43:05.186808 15604 ts_tablet_manager.cc:1905] T 95ca5e1f18a443c2a69c7ca9affac919 P 221d138c3adb48d6b239f506a8aaa778: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:43:05.186976 15606 raft_consensus.cc:2258] T 95ca5e1f18a443c2a69c7ca9affac919 P c22e9dc4838648d2bdaaa76af7ffc4b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:43:05.189741 15604 ts_tablet_manager.cc:1918] T 95ca5e1f18a443c2a69c7ca9affac919 P 221d138c3adb48d6b239f506a8aaa778: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 02:43:05.189927 15604 log.cc:1196] T 95ca5e1f18a443c2a69c7ca9affac919 P 221d138c3adb48d6b239f506a8aaa778: Deleting WAL directory at /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790304178325258-13942-0/minicluster-data/ts-0/wal/wals/95ca5e1f18a443c2a69c7ca9affac919
I20260925 02:43:05.190267 15604 ts_tablet_manager.cc:1939] T 95ca5e1f18a443c2a69c7ca9affac919 P 221d138c3adb48d6b239f506a8aaa778: Deleting consensus metadata
I20260925 02:43:05.190980 15605 ts_tablet_manager.cc:1905] T 95ca5e1f18a443c2a69c7ca9affac919 P 7d639c741543473587167bd497e3fc7d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:43:05.190951 15344 tablet_service.cc:1467] Processing CreateTablet for tablet f6fd9caa5f12412db461cdf94b7d04dc (DEFAULT_TABLE table=test-table [id=35213401a781460e9821acee4d9646a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:43:05.191239 15344 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f6fd9caa5f12412db461cdf94b7d04dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:43:05.191409 15202 tablet_service.cc:1467] Processing CreateTablet for tablet f6fd9caa5f12412db461cdf94b7d04dc (DEFAULT_TABLE table=test-table [id=35213401a781460e9821acee4d9646a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:43:05.191649 15202 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f6fd9caa5f12412db461cdf94b7d04dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:43:05.193626 15582 tablet_bootstrap.cc:492] T f6fd9caa5f12412db461cdf94b7d04dc P 7d639c741543473587167bd497e3fc7d: Bootstrap starting.
I20260925 02:43:05.194178 15581 tablet_bootstrap.cc:492] T f6fd9caa5f12412db461cdf94b7d04dc P 221d138c3adb48d6b239f506a8aaa778: Bootstrap starting.
I20260925 02:43:05.194756 15582 tablet_bootstrap.cc:654] T f6fd9caa5f12412db461cdf94b7d04dc P 7d639c741543473587167bd497e3fc7d: Neither blocks nor log segments found. Creating new log.
I20260925 02:43:05.195117 15581 tablet_bootstrap.cc:654] T f6fd9caa5f12412db461cdf94b7d04dc P 221d138c3adb48d6b239f506a8aaa778: Neither blocks nor log segments found. Creating new log.
I20260925 02:43:05.196568 15046 catalog_manager.cc:4841] TS 221d138c3adb48d6b239f506a8aaa778 (127.13.157.129:39953): tablet 95ca5e1f18a443c2a69c7ca9affac919 (table test-table [id=aea6c7844041492b9ebc813f14bc3201]) successfully deleted
I20260925 02:43:05.196656 15488 tablet_service.cc:1467] Processing CreateTablet for tablet f6fd9caa5f12412db461cdf94b7d04dc (DEFAULT_TABLE table=test-table [id=35213401a781460e9821acee4d9646a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:43:05.196741 15605 ts_tablet_manager.cc:1918] T 95ca5e1f18a443c2a69c7ca9affac919 P 7d639c741543473587167bd497e3fc7d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 02:43:05.196823 15605 log.cc:1196] T 95ca5e1f18a443c2a69c7ca9affac919 P 7d639c741543473587167bd497e3fc7d: Deleting WAL directory at /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790304178325258-13942-0/minicluster-data/ts-1/wal/wals/95ca5e1f18a443c2a69c7ca9affac919
I20260925 02:43:05.196923 15488 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f6fd9caa5f12412db461cdf94b7d04dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:43:05.197118 15605 ts_tablet_manager.cc:1939] T 95ca5e1f18a443c2a69c7ca9affac919 P 7d639c741543473587167bd497e3fc7d: Deleting consensus metadata
I20260925 02:43:05.198832 15049 catalog_manager.cc:4841] TS 7d639c741543473587167bd497e3fc7d (127.13.157.130:37769): tablet 95ca5e1f18a443c2a69c7ca9affac919 (table test-table [id=aea6c7844041492b9ebc813f14bc3201]) successfully deleted
I20260925 02:43:05.199177 15606 ts_tablet_manager.cc:1905] T 95ca5e1f18a443c2a69c7ca9affac919 P c22e9dc4838648d2bdaaa76af7ffc4b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:43:05.200073 15580 tablet_bootstrap.cc:492] T f6fd9caa5f12412db461cdf94b7d04dc P c22e9dc4838648d2bdaaa76af7ffc4b7: Bootstrap starting.
I20260925 02:43:05.200871 15581 tablet_bootstrap.cc:492] T f6fd9caa5f12412db461cdf94b7d04dc P 221d138c3adb48d6b239f506a8aaa778: No bootstrap required, opened a new log
I20260925 02:43:05.200963 15581 ts_tablet_manager.cc:1397] T f6fd9caa5f12412db461cdf94b7d04dc P 221d138c3adb48d6b239f506a8aaa778: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20260925 02:43:05.201459 15580 tablet_bootstrap.cc:654] T f6fd9caa5f12412db461cdf94b7d04dc P c22e9dc4838648d2bdaaa76af7ffc4b7: Neither blocks nor log segments found. Creating new log.
I20260925 02:43:05.201642 15581 raft_consensus.cc:348] T f6fd9caa5f12412db461cdf94b7d04dc P 221d138c3adb48d6b239f506a8aaa778 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "221d138c3adb48d6b239f506a8aaa778" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 39953 } } peers { permanent_uuid: "7d639c741543473587167bd497e3fc7d" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 37769 } } peers { permanent_uuid: "c22e9dc4838648d2bdaaa76af7ffc4b7" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 45991 } }
I20260925 02:43:05.201759 15581 raft_consensus.cc:374] T f6fd9caa5f12412db461cdf94b7d04dc P 221d138c3adb48d6b239f506a8aaa778 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:43:05.201761 15606 ts_tablet_manager.cc:1918] T 95ca5e1f18a443c2a69c7ca9affac919 P c22e9dc4838648d2bdaaa76af7ffc4b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 02:43:05.201793 15581 raft_consensus.cc:729] T f6fd9caa5f12412db461cdf94b7d04dc P 221d138c3adb48d6b239f506a8aaa778 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 221d138c3adb48d6b239f506a8aaa778, State: Initialized, Role: FOLLOWER
I20260925 02:43:05.201839 15606 log.cc:1196] T 95ca5e1f18a443c2a69c7ca9affac919 P c22e9dc4838648d2bdaaa76af7ffc4b7: Deleting WAL directory at /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790304178325258-13942-0/minicluster-data/ts-2/wal/wals/95ca5e1f18a443c2a69c7ca9affac919
I20260925 02:43:05.201954 15581 consensus_queue.cc:260] T f6fd9caa5f12412db461cdf94b7d04dc P 221d138c3adb48d6b239f506a8aaa778 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "221d138c3adb48d6b239f506a8aaa778" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 39953 } } peers { permanent_uuid: "7d639c741543473587167bd497e3fc7d" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 37769 } } peers { permanent_uuid: "c22e9dc4838648d2bdaaa76af7ffc4b7" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 45991 } }
I20260925 02:43:05.202122 15606 ts_tablet_manager.cc:1939] T 95ca5e1f18a443c2a69c7ca9affac919 P c22e9dc4838648d2bdaaa76af7ffc4b7: Deleting consensus metadata
I20260925 02:43:05.202845 15581 ts_tablet_manager.cc:1428] T f6fd9caa5f12412db461cdf94b7d04dc P 221d138c3adb48d6b239f506a8aaa778: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 02:43:05.203716 15582 tablet_bootstrap.cc:492] T f6fd9caa5f12412db461cdf94b7d04dc P 7d639c741543473587167bd497e3fc7d: No bootstrap required, opened a new log
I20260925 02:43:05.203797 15582 ts_tablet_manager.cc:1397] T f6fd9caa5f12412db461cdf94b7d04dc P 7d639c741543473587167bd497e3fc7d: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.002s
I20260925 02:43:05.204262 15582 raft_consensus.cc:348] T f6fd9caa5f12412db461cdf94b7d04dc P 7d639c741543473587167bd497e3fc7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "221d138c3adb48d6b239f506a8aaa778" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 39953 } } peers { permanent_uuid: "7d639c741543473587167bd497e3fc7d" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 37769 } } peers { permanent_uuid: "c22e9dc4838648d2bdaaa76af7ffc4b7" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 45991 } }
I20260925 02:43:05.204367 15582 raft_consensus.cc:374] T f6fd9caa5f12412db461cdf94b7d04dc P 7d639c741543473587167bd497e3fc7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:43:05.204401 15582 raft_consensus.cc:729] T f6fd9caa5f12412db461cdf94b7d04dc P 7d639c741543473587167bd497e3fc7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d639c741543473587167bd497e3fc7d, State: Initialized, Role: FOLLOWER
I20260925 02:43:05.204516 15582 consensus_queue.cc:260] T f6fd9caa5f12412db461cdf94b7d04dc P 7d639c741543473587167bd497e3fc7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "221d138c3adb48d6b239f506a8aaa778" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 39953 } } peers { permanent_uuid: "7d639c741543473587167bd497e3fc7d" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 37769 } } peers { permanent_uuid: "c22e9dc4838648d2bdaaa76af7ffc4b7" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 45991 } }
I20260925 02:43:05.204769 15582 ts_tablet_manager.cc:1428] T f6fd9caa5f12412db461cdf94b7d04dc P 7d639c741543473587167bd497e3fc7d: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260925 02:43:05.205286 15049 catalog_manager.cc:4841] TS c22e9dc4838648d2bdaaa76af7ffc4b7 (127.13.157.131:45991): tablet 95ca5e1f18a443c2a69c7ca9affac919 (table test-table [id=aea6c7844041492b9ebc813f14bc3201]) successfully deleted
I20260925 02:43:05.206046 15580 tablet_bootstrap.cc:492] T f6fd9caa5f12412db461cdf94b7d04dc P c22e9dc4838648d2bdaaa76af7ffc4b7: No bootstrap required, opened a new log
I20260925 02:43:05.206122 15580 ts_tablet_manager.cc:1397] T f6fd9caa5f12412db461cdf94b7d04dc P c22e9dc4838648d2bdaaa76af7ffc4b7: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260925 02:43:05.206584 15580 raft_consensus.cc:348] T f6fd9caa5f12412db461cdf94b7d04dc P c22e9dc4838648d2bdaaa76af7ffc4b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "221d138c3adb48d6b239f506a8aaa778" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 39953 } } peers { permanent_uuid: "7d639c741543473587167bd497e3fc7d" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 37769 } } peers { permanent_uuid: "c22e9dc4838648d2bdaaa76af7ffc4b7" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 45991 } }
I20260925 02:43:05.206676 15580 raft_consensus.cc:374] T f6fd9caa5f12412db461cdf94b7d04dc P c22e9dc4838648d2bdaaa76af7ffc4b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:43:05.206704 15580 raft_consensus.cc:729] T f6fd9caa5f12412db461cdf94b7d04dc P c22e9dc4838648d2bdaaa76af7ffc4b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c22e9dc4838648d2bdaaa76af7ffc4b7, State: Initialized, Role: FOLLOWER
I20260925 02:43:05.206823 15580 consensus_queue.cc:260] T f6fd9caa5f12412db461cdf94b7d04dc P c22e9dc4838648d2bdaaa76af7ffc4b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "221d138c3adb48d6b239f506a8aaa778" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 39953 } } peers { permanent_uuid: "7d639c741543473587167bd497e3fc7d" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 37769 } } peers { permanent_uuid: "c22e9dc4838648d2bdaaa76af7ffc4b7" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 45991 } }
I20260925 02:43:05.207103 15580 ts_tablet_manager.cc:1428] T f6fd9caa5f12412db461cdf94b7d04dc P c22e9dc4838648d2bdaaa76af7ffc4b7: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
W20260925 02:43:05.233158 15412 tablet.cc:2369] T f6fd9caa5f12412db461cdf94b7d04dc P 7d639c741543473587167bd497e3fc7d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 02:43:05.236763 15587 raft_consensus.cc:482] T f6fd9caa5f12412db461cdf94b7d04dc P 7d639c741543473587167bd497e3fc7d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 02:43:05.236897 15587 raft_consensus.cc:504] T f6fd9caa5f12412db461cdf94b7d04dc P 7d639c741543473587167bd497e3fc7d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "221d138c3adb48d6b239f506a8aaa778" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 39953 } } peers { permanent_uuid: "7d639c741543473587167bd497e3fc7d" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 37769 } } peers { permanent_uuid: "c22e9dc4838648d2bdaaa76af7ffc4b7" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 45991 } }
I20260925 02:43:05.238183 15587 leader_election.cc:288] T f6fd9caa5f12412db461cdf94b7d04dc P 7d639c741543473587167bd497e3fc7d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 221d138c3adb48d6b239f506a8aaa778 (127.13.157.129:39953), c22e9dc4838648d2bdaaa76af7ffc4b7 (127.13.157.131:45991)
I20260925 02:43:05.243674 15577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:05.239264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.130:60701 (local address 127.13.157.129:39953)
0925 02:43:05.239417 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:05.239422 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:05.239435 (+ 13us) server_negotiation.cc:407] Connection header received
0925 02:43:05.239484 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:05.239487 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:05.239529 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:05.239588 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:43:05.239963 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:05.240700 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:05.242358 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:05.242974 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:05.243098 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:05.243152 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:05.243471 (+ 319us) server_negotiation.cc:299] Negotiation successful
0925 02:43:05.243541 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260925 02:43:05.244065 15579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:05.238963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.130:47859 (local address 127.13.157.131:45991)
0925 02:43:05.239083 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:05.239087 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:05.239521 (+ 434us) server_negotiation.cc:407] Connection header received
0925 02:43:05.239705 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:05.239708 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:05.239755 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:05.239814 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:43:05.240305 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:05.241062 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:05.242771 (+ 1709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:05.243398 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:05.243487 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:05.243538 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:05.243873 (+ 335us) server_negotiation.cc:299] Negotiation successful
0925 02:43:05.243942 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260925 02:43:05.244436 15610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:05.238364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.129:39953 (local address 127.13.157.130:60701)
0925 02:43:05.238804 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:05.238818 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:05.238936 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:05.239695 (+ 759us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:05.239699 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:05.239716 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:43:05.239844 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:05.239852 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:05.240837 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:05.240840 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:05.242212 (+ 1372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:05.242222 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:05.243244 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:05.243247 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:05.243349 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:05.243371 (+ 22us) client_negotiation.cc:769] Sending connection context
0925 02:43:05.244047 (+ 676us) client_negotiation.cc:240] Negotiation successful
0925 02:43:05.244288 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":80,"threads_started":1}
I20260925 02:43:05.244494 15611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:05.238491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.131:45991 (local address 127.13.157.130:47859)
0925 02:43:05.239437 (+ 946us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:05.239450 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:05.239593 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:05.239941 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:05.239944 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:05.239960 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:43:05.240173 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:05.240181 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:05.241210 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:05.241213 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:05.242632 (+ 1419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:05.242643 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:05.243645 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:05.243649 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:05.243746 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:05.243766 (+ 20us) client_negotiation.cc:769] Sending connection context
0925 02:43:05.244280 (+ 514us) client_negotiation.cc:240] Negotiation successful
0925 02:43:05.244357 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":880,"thread_start_us":88,"threads_started":1}
I20260925 02:43:05.245239 15223 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6fd9caa5f12412db461cdf94b7d04dc" candidate_uuid: "7d639c741543473587167bd497e3fc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "221d138c3adb48d6b239f506a8aaa778" is_pre_election: true
I20260925 02:43:05.245414 15223 raft_consensus.cc:2454] T f6fd9caa5f12412db461cdf94b7d04dc P 221d138c3adb48d6b239f506a8aaa778 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d639c741543473587167bd497e3fc7d in term 0.
I20260925 02:43:05.245303 15508 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6fd9caa5f12412db461cdf94b7d04dc" candidate_uuid: "7d639c741543473587167bd497e3fc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c22e9dc4838648d2bdaaa76af7ffc4b7" is_pre_election: true
I20260925 02:43:05.245601 15508 raft_consensus.cc:2454] T f6fd9caa5f12412db461cdf94b7d04dc P c22e9dc4838648d2bdaaa76af7ffc4b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d639c741543473587167bd497e3fc7d in term 0.
I20260925 02:43:05.245893 15295 leader_election.cc:302] T f6fd9caa5f12412db461cdf94b7d04dc P 7d639c741543473587167bd497e3fc7d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 221d138c3adb48d6b239f506a8aaa778, 7d639c741543473587167bd497e3fc7d; no voters:
I20260925 02:43:05.246196 15587 raft_consensus.cc:2781] T f6fd9caa5f12412db461cdf94b7d04dc P 7d639c741543473587167bd497e3fc7d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 02:43:05.246259 15587 raft_consensus.cc:482] T f6fd9caa5f12412db461cdf94b7d04dc P 7d639c741543473587167bd497e3fc7d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 02:43:05.246295 15587 raft_consensus.cc:3037] T f6fd9caa5f12412db461cdf94b7d04dc P 7d639c741543473587167bd497e3fc7d [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:43:05.247779 15587 raft_consensus.cc:504] T f6fd9caa5f12412db461cdf94b7d04dc P 7d639c741543473587167bd497e3fc7d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "221d138c3adb48d6b239f506a8aaa778" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 39953 } } peers { permanent_uuid: "7d639c741543473587167bd497e3fc7d" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 37769 } } peers { permanent_uuid: "c22e9dc4838648d2bdaaa76af7ffc4b7" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 45991 } }
I20260925 02:43:05.248112 15587 leader_election.cc:288] T f6fd9caa5f12412db461cdf94b7d04dc P 7d639c741543473587167bd497e3fc7d [CANDIDATE]: Term 1 election: Requested vote from peers 221d138c3adb48d6b239f506a8aaa778 (127.13.157.129:39953), c22e9dc4838648d2bdaaa76af7ffc4b7 (127.13.157.131:45991)
I20260925 02:43:05.248435 15508 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6fd9caa5f12412db461cdf94b7d04dc" candidate_uuid: "7d639c741543473587167bd497e3fc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c22e9dc4838648d2bdaaa76af7ffc4b7"
I20260925 02:43:05.248452 15223 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6fd9caa5f12412db461cdf94b7d04dc" candidate_uuid: "7d639c741543473587167bd497e3fc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "221d138c3adb48d6b239f506a8aaa778"
I20260925 02:43:05.248555 15508 raft_consensus.cc:3037] T f6fd9caa5f12412db461cdf94b7d04dc P c22e9dc4838648d2bdaaa76af7ffc4b7 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:43:05.248574 15223 raft_consensus.cc:3037] T f6fd9caa5f12412db461cdf94b7d04dc P 221d138c3adb48d6b239f506a8aaa778 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:43:05.249660 15508 raft_consensus.cc:2454] T f6fd9caa5f12412db461cdf94b7d04dc P c22e9dc4838648d2bdaaa76af7ffc4b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d639c741543473587167bd497e3fc7d in term 1.
I20260925 02:43:05.249934 15298 leader_election.cc:302] T f6fd9caa5f12412db461cdf94b7d04dc P 7d639c741543473587167bd497e3fc7d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d639c741543473587167bd497e3fc7d, c22e9dc4838648d2bdaaa76af7ffc4b7; no voters:
I20260925 02:43:05.250056 15223 raft_consensus.cc:2454] T f6fd9caa5f12412db461cdf94b7d04dc P 221d138c3adb48d6b239f506a8aaa778 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d639c741543473587167bd497e3fc7d in term 1.
I20260925 02:43:05.250092 15587 raft_consensus.cc:2781] T f6fd9caa5f12412db461cdf94b7d04dc P 7d639c741543473587167bd497e3fc7d [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:43:05.250252 15587 raft_consensus.cc:686] T f6fd9caa5f12412db461cdf94b7d04dc P 7d639c741543473587167bd497e3fc7d [term 1 LEADER]: Becoming Leader. State: Replica: 7d639c741543473587167bd497e3fc7d, State: Running, Role: LEADER
I20260925 02:43:05.250495 15587 consensus_queue.cc:237] T f6fd9caa5f12412db461cdf94b7d04dc P 7d639c741543473587167bd497e3fc7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "221d138c3adb48d6b239f506a8aaa778" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 39953 } } peers { permanent_uuid: "7d639c741543473587167bd497e3fc7d" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 37769 } } peers { permanent_uuid: "c22e9dc4838648d2bdaaa76af7ffc4b7" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 45991 } }
I20260925 02:43:05.252842 15062 catalog_manager.cc:5495] T f6fd9caa5f12412db461cdf94b7d04dc P 7d639c741543473587167bd497e3fc7d reported cstate change: term changed from 0 to 1, leader changed from <none> to 7d639c741543473587167bd497e3fc7d (127.13.157.130). New cstate: current_term: 1 leader_uuid: "7d639c741543473587167bd497e3fc7d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "221d138c3adb48d6b239f506a8aaa778" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 39953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d639c741543473587167bd497e3fc7d" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 37769 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c22e9dc4838648d2bdaaa76af7ffc4b7" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 45991 } health_report { overall_health: UNKNOWN } } }
Sep 25 02:43:05 dist-test-slave-kznw krb5kdc[15017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304184, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.190@KRBTEST.COM for kudu/127.13.157.190@KRBTEST.COM
I20260925 02:43:05.285537 15113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:05.276616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40544 (local address 127.13.157.190:32809)
0925 02:43:05.276758 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:05.276764 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:05.276944 (+ 180us) server_negotiation.cc:407] Connection header received
0925 02:43:05.277086 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:05.277088 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:05.277130 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:05.277197 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:05.277990 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:05.278522 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:05.279612 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:05.279847 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:05.279928 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:05.282491 (+ 2563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:05.282494 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:05.282496 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:43:05.282518 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:05.283824 (+ 1306us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:05.284328 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:05.284331 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:05.284332 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:05.284396 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:05.284646 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:05.284648 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:05.284649 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:05.284823 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:05.284894 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:05.285215 (+ 321us) server_negotiation.cc:299] Negotiation successful
0925 02:43:05.285385 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":44,"server-negotiator.queue_time_us":55}
I20260925 02:43:05.285553 15619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:05.276531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:32809 (local address 127.0.0.1:40544)
0925 02:43:05.276852 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:05.276868 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:05.276991 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:05.277243 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:05.277246 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:05.277746 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:05.277870 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:05.277879 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:05.278630 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:05.278633 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:05.279505 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:05.279517 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:05.279957 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:05.279961 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:05.280061 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:05.280706 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:05.280722 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:05.282359 (+ 1637us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:43:05.283927 (+ 1568us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:05.283937 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:05.283942 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:05.284244 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:05.284467 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:05.284470 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:05.284472 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:05.284570 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:05.284917 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:43:05.284923 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:43:05.285044 (+ 121us) client_negotiation.cc:769] Sending connection context
0925 02:43:05.285169 (+ 125us) client_negotiation.cc:240] Negotiation successful
0925 02:43:05.285322 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":95,"threads_started":1}
I20260925 02:43:05.289851 15062 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.13.157.190@KRBTEST.COM'} at 127.0.0.1:40544:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260925 02:43:05.290587 15062 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 02:43:05.295519 15202 tablet_service.cc:1467] Processing CreateTablet for tablet ee66e1f6d8684d618a60cba6e523e0e1 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c429cf9d03684037a825a4b5b222f1c1]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 02:43:05.295977 15202 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee66e1f6d8684d618a60cba6e523e0e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:43:05.295946 15344 tablet_service.cc:1467] Processing CreateTablet for tablet ee66e1f6d8684d618a60cba6e523e0e1 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c429cf9d03684037a825a4b5b222f1c1]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 02:43:05.296300 15344 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee66e1f6d8684d618a60cba6e523e0e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:43:05.299381 15581 tablet_bootstrap.cc:492] T ee66e1f6d8684d618a60cba6e523e0e1 P 221d138c3adb48d6b239f506a8aaa778: Bootstrap starting.
I20260925 02:43:05.299935 15582 tablet_bootstrap.cc:492] T ee66e1f6d8684d618a60cba6e523e0e1 P 7d639c741543473587167bd497e3fc7d: Bootstrap starting.
I20260925 02:43:05.300518 15581 tablet_bootstrap.cc:654] T ee66e1f6d8684d618a60cba6e523e0e1 P 221d138c3adb48d6b239f506a8aaa778: Neither blocks nor log segments found. Creating new log.
I20260925 02:43:05.300985 15582 tablet_bootstrap.cc:654] T ee66e1f6d8684d618a60cba6e523e0e1 P 7d639c741543473587167bd497e3fc7d: Neither blocks nor log segments found. Creating new log.
I20260925 02:43:05.301049 15488 tablet_service.cc:1467] Processing CreateTablet for tablet ee66e1f6d8684d618a60cba6e523e0e1 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c429cf9d03684037a825a4b5b222f1c1]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 02:43:05.301403 15488 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee66e1f6d8684d618a60cba6e523e0e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:43:05.302383 15582 tablet_bootstrap.cc:492] T ee66e1f6d8684d618a60cba6e523e0e1 P 7d639c741543473587167bd497e3fc7d: No bootstrap required, opened a new log
I20260925 02:43:05.302476 15582 ts_tablet_manager.cc:1397] T ee66e1f6d8684d618a60cba6e523e0e1 P 7d639c741543473587167bd497e3fc7d: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260925 02:43:05.302999 15582 raft_consensus.cc:348] T ee66e1f6d8684d618a60cba6e523e0e1 P 7d639c741543473587167bd497e3fc7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "221d138c3adb48d6b239f506a8aaa778" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 39953 } } peers { permanent_uuid: "7d639c741543473587167bd497e3fc7d" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 37769 } } peers { permanent_uuid: "c22e9dc4838648d2bdaaa76af7ffc4b7" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 45991 } }
I20260925 02:43:05.303226 15582 raft_consensus.cc:374] T ee66e1f6d8684d618a60cba6e523e0e1 P 7d639c741543473587167bd497e3fc7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:43:05.303283 15582 raft_consensus.cc:729] T ee66e1f6d8684d618a60cba6e523e0e1 P 7d639c741543473587167bd497e3fc7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d639c741543473587167bd497e3fc7d, State: Initialized, Role: FOLLOWER
I20260925 02:43:05.303421 15582 consensus_queue.cc:260] T ee66e1f6d8684d618a60cba6e523e0e1 P 7d639c741543473587167bd497e3fc7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "221d138c3adb48d6b239f506a8aaa778" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 39953 } } peers { permanent_uuid: "7d639c741543473587167bd497e3fc7d" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 37769 } } peers { permanent_uuid: "c22e9dc4838648d2bdaaa76af7ffc4b7" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 45991 } }
I20260925 02:43:05.303664 15587 tablet_replica.cc:439] T ee66e1f6d8684d618a60cba6e523e0e1 P 7d639c741543473587167bd497e3fc7d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "221d138c3adb48d6b239f506a8aaa778" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 39953 } } peers { permanent_uuid: "7d639c741543473587167bd497e3fc7d" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 37769 } } peers { permanent_uuid: "c22e9dc4838648d2bdaaa76af7ffc4b7" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 45991 } } }
I20260925 02:43:05.303793 15587 tablet_replica.cc:442] T ee66e1f6d8684d618a60cba6e523e0e1 P 7d639c741543473587167bd497e3fc7d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 02:43:05.303928 15580 tablet_bootstrap.cc:492] T ee66e1f6d8684d618a60cba6e523e0e1 P c22e9dc4838648d2bdaaa76af7ffc4b7: Bootstrap starting.
I20260925 02:43:05.304026 15582 ts_tablet_manager.cc:1428] T ee66e1f6d8684d618a60cba6e523e0e1 P 7d639c741543473587167bd497e3fc7d: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260925 02:43:05.304828 15580 tablet_bootstrap.cc:654] T ee66e1f6d8684d618a60cba6e523e0e1 P c22e9dc4838648d2bdaaa76af7ffc4b7: Neither blocks nor log segments found. Creating new log.
I20260925 02:43:05.305972 15580 tablet_bootstrap.cc:492] T ee66e1f6d8684d618a60cba6e523e0e1 P c22e9dc4838648d2bdaaa76af7ffc4b7: No bootstrap required, opened a new log
I20260925 02:43:05.306051 15580 ts_tablet_manager.cc:1397] T ee66e1f6d8684d618a60cba6e523e0e1 P c22e9dc4838648d2bdaaa76af7ffc4b7: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 02:43:05.306733 15580 raft_consensus.cc:348] T ee66e1f6d8684d618a60cba6e523e0e1 P c22e9dc4838648d2bdaaa76af7ffc4b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "221d138c3adb48d6b239f506a8aaa778" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 39953 } } peers { permanent_uuid: "7d639c741543473587167bd497e3fc7d" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 37769 } } peers { permanent_uuid: "c22e9dc4838648d2bdaaa76af7ffc4b7" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 45991 } }
I20260925 02:43:05.306851 15580 raft_consensus.cc:374] T ee66e1f6d8684d618a60cba6e523e0e1 P c22e9dc4838648d2bdaaa76af7ffc4b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:43:05.306885 15580 raft_consensus.cc:729] T ee66e1f6d8684d618a60cba6e523e0e1 P c22e9dc4838648d2bdaaa76af7ffc4b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c22e9dc4838648d2bdaaa76af7ffc4b7, State: Initialized, Role: FOLLOWER
I20260925 02:43:05.307029 15580 consensus_queue.cc:260] T ee66e1f6d8684d618a60cba6e523e0e1 P c22e9dc4838648d2bdaaa76af7ffc4b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "221d138c3adb48d6b239f506a8aaa778" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 39953 } } peers { permanent_uuid: "7d639c741543473587167bd497e3fc7d" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 37769 } } peers { permanent_uuid: "c22e9dc4838648d2bdaaa76af7ffc4b7" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 45991 } }
I20260925 02:43:05.307299 15580 ts_tablet_manager.cc:1428] T ee66e1f6d8684d618a60cba6e523e0e1 P c22e9dc4838648d2bdaaa76af7ffc4b7: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260925 02:43:05.307261 15599 tablet_replica.cc:439] T ee66e1f6d8684d618a60cba6e523e0e1 P c22e9dc4838648d2bdaaa76af7ffc4b7: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "221d138c3adb48d6b239f506a8aaa778" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 39953 } } peers { permanent_uuid: "7d639c741543473587167bd497e3fc7d" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 37769 } } peers { permanent_uuid: "c22e9dc4838648d2bdaaa76af7ffc4b7" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 45991 } } }
I20260925 02:43:05.307394 15599 tablet_replica.cc:442] T ee66e1f6d8684d618a60cba6e523e0e1 P c22e9dc4838648d2bdaaa76af7ffc4b7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 02:43:05.308522 15581 tablet_bootstrap.cc:492] T ee66e1f6d8684d618a60cba6e523e0e1 P 221d138c3adb48d6b239f506a8aaa778: No bootstrap required, opened a new log
I20260925 02:43:05.308632 15581 ts_tablet_manager.cc:1397] T ee66e1f6d8684d618a60cba6e523e0e1 P 221d138c3adb48d6b239f506a8aaa778: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20260925 02:43:05.309335 15581 raft_consensus.cc:348] T ee66e1f6d8684d618a60cba6e523e0e1 P 221d138c3adb48d6b239f506a8aaa778 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "221d138c3adb48d6b239f506a8aaa778" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 39953 } } peers { permanent_uuid: "7d639c741543473587167bd497e3fc7d" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 37769 } } peers { permanent_uuid: "c22e9dc4838648d2bdaaa76af7ffc4b7" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 45991 } }
I20260925 02:43:05.309433 15581 raft_consensus.cc:374] T ee66e1f6d8684d618a60cba6e523e0e1 P 221d138c3adb48d6b239f506a8aaa778 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:43:05.309454 15581 raft_consensus.cc:729] T ee66e1f6d8684d618a60cba6e523e0e1 P 221d138c3adb48d6b239f506a8aaa778 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 221d138c3adb48d6b239f506a8aaa778, State: Initialized, Role: FOLLOWER
I20260925 02:43:05.309561 15581 consensus_queue.cc:260] T ee66e1f6d8684d618a60cba6e523e0e1 P 221d138c3adb48d6b239f506a8aaa778 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "221d138c3adb48d6b239f506a8aaa778" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 39953 } } peers { permanent_uuid: "7d639c741543473587167bd497e3fc7d" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 37769 } } peers { permanent_uuid: "c22e9dc4838648d2bdaaa76af7ffc4b7" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 45991 } }
I20260925 02:43:05.309800 15581 ts_tablet_manager.cc:1428] T ee66e1f6d8684d618a60cba6e523e0e1 P 221d138c3adb48d6b239f506a8aaa778: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260925 02:43:05.309811 15589 tablet_replica.cc:439] T ee66e1f6d8684d618a60cba6e523e0e1 P 221d138c3adb48d6b239f506a8aaa778: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "221d138c3adb48d6b239f506a8aaa778" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 39953 } } peers { permanent_uuid: "7d639c741543473587167bd497e3fc7d" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 37769 } } peers { permanent_uuid: "c22e9dc4838648d2bdaaa76af7ffc4b7" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 45991 } } }
I20260925 02:43:05.310318 15589 tablet_replica.cc:442] T ee66e1f6d8684d618a60cba6e523e0e1 P 221d138c3adb48d6b239f506a8aaa778: This TxnStatusTablet replica's current role is: FOLLOWER
W20260925 02:43:05.442749 15557 tablet.cc:2369] T ee66e1f6d8684d618a60cba6e523e0e1 P c22e9dc4838648d2bdaaa76af7ffc4b7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 02:43:05.525540 15599 raft_consensus.cc:482] T ee66e1f6d8684d618a60cba6e523e0e1 P c22e9dc4838648d2bdaaa76af7ffc4b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 02:43:05.525687 15599 raft_consensus.cc:504] T ee66e1f6d8684d618a60cba6e523e0e1 P c22e9dc4838648d2bdaaa76af7ffc4b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "221d138c3adb48d6b239f506a8aaa778" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 39953 } } peers { permanent_uuid: "7d639c741543473587167bd497e3fc7d" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 37769 } } peers { permanent_uuid: "c22e9dc4838648d2bdaaa76af7ffc4b7" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 45991 } }
I20260925 02:43:05.526108 15599 leader_election.cc:288] T ee66e1f6d8684d618a60cba6e523e0e1 P c22e9dc4838648d2bdaaa76af7ffc4b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 221d138c3adb48d6b239f506a8aaa778 (127.13.157.129:39953), 7d639c741543473587167bd497e3fc7d (127.13.157.130:37769)
I20260925 02:43:05.526582 15223 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee66e1f6d8684d618a60cba6e523e0e1" candidate_uuid: "c22e9dc4838648d2bdaaa76af7ffc4b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "221d138c3adb48d6b239f506a8aaa778" is_pre_election: true
I20260925 02:43:05.526619 15364 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee66e1f6d8684d618a60cba6e523e0e1" candidate_uuid: "c22e9dc4838648d2bdaaa76af7ffc4b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d639c741543473587167bd497e3fc7d" is_pre_election: true
I20260925 02:43:05.526729 15223 raft_consensus.cc:2454] T ee66e1f6d8684d618a60cba6e523e0e1 P 221d138c3adb48d6b239f506a8aaa778 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c22e9dc4838648d2bdaaa76af7ffc4b7 in term 0.
I20260925 02:43:05.526752 15364 raft_consensus.cc:2454] T ee66e1f6d8684d618a60cba6e523e0e1 P 7d639c741543473587167bd497e3fc7d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c22e9dc4838648d2bdaaa76af7ffc4b7 in term 0.
I20260925 02:43:05.527041 15438 leader_election.cc:302] T ee66e1f6d8684d618a60cba6e523e0e1 P c22e9dc4838648d2bdaaa76af7ffc4b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 221d138c3adb48d6b239f506a8aaa778, c22e9dc4838648d2bdaaa76af7ffc4b7; no voters:
I20260925 02:43:05.527272 15599 raft_consensus.cc:2781] T ee66e1f6d8684d618a60cba6e523e0e1 P c22e9dc4838648d2bdaaa76af7ffc4b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 02:43:05.527326 15599 raft_consensus.cc:482] T ee66e1f6d8684d618a60cba6e523e0e1 P c22e9dc4838648d2bdaaa76af7ffc4b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 02:43:05.527374 15599 raft_consensus.cc:3037] T ee66e1f6d8684d618a60cba6e523e0e1 P c22e9dc4838648d2bdaaa76af7ffc4b7 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:43:05.528409 15599 raft_consensus.cc:504] T ee66e1f6d8684d618a60cba6e523e0e1 P c22e9dc4838648d2bdaaa76af7ffc4b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "221d138c3adb48d6b239f506a8aaa778" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 39953 } } peers { permanent_uuid: "7d639c741543473587167bd497e3fc7d" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 37769 } } peers { permanent_uuid: "c22e9dc4838648d2bdaaa76af7ffc4b7" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 45991 } }
I20260925 02:43:05.528733 15599 leader_election.cc:288] T ee66e1f6d8684d618a60cba6e523e0e1 P c22e9dc4838648d2bdaaa76af7ffc4b7 [CANDIDATE]: Term 1 election: Requested vote from peers 221d138c3adb48d6b239f506a8aaa778 (127.13.157.129:39953), 7d639c741543473587167bd497e3fc7d (127.13.157.130:37769)
I20260925 02:43:05.529040 15223 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee66e1f6d8684d618a60cba6e523e0e1" candidate_uuid: "c22e9dc4838648d2bdaaa76af7ffc4b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "221d138c3adb48d6b239f506a8aaa778"
I20260925 02:43:05.529072 15364 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee66e1f6d8684d618a60cba6e523e0e1" candidate_uuid: "c22e9dc4838648d2bdaaa76af7ffc4b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d639c741543473587167bd497e3fc7d"
I20260925 02:43:05.529156 15223 raft_consensus.cc:3037] T ee66e1f6d8684d618a60cba6e523e0e1 P 221d138c3adb48d6b239f506a8aaa778 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:43:05.529179 15364 raft_consensus.cc:3037] T ee66e1f6d8684d618a60cba6e523e0e1 P 7d639c741543473587167bd497e3fc7d [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:43:05.530248 15223 raft_consensus.cc:2454] T ee66e1f6d8684d618a60cba6e523e0e1 P 221d138c3adb48d6b239f506a8aaa778 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c22e9dc4838648d2bdaaa76af7ffc4b7 in term 1.
I20260925 02:43:05.530455 15364 raft_consensus.cc:2454] T ee66e1f6d8684d618a60cba6e523e0e1 P 7d639c741543473587167bd497e3fc7d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c22e9dc4838648d2bdaaa76af7ffc4b7 in term 1.
I20260925 02:43:05.530721 15441 leader_election.cc:302] T ee66e1f6d8684d618a60cba6e523e0e1 P c22e9dc4838648d2bdaaa76af7ffc4b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d639c741543473587167bd497e3fc7d, c22e9dc4838648d2bdaaa76af7ffc4b7; no voters:
I20260925 02:43:05.530994 15599 raft_consensus.cc:2781] T ee66e1f6d8684d618a60cba6e523e0e1 P c22e9dc4838648d2bdaaa76af7ffc4b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:43:05.531088 15599 raft_consensus.cc:686] T ee66e1f6d8684d618a60cba6e523e0e1 P c22e9dc4838648d2bdaaa76af7ffc4b7 [term 1 LEADER]: Becoming Leader. State: Replica: c22e9dc4838648d2bdaaa76af7ffc4b7, State: Running, Role: LEADER
I20260925 02:43:05.531225 15599 consensus_queue.cc:237] T ee66e1f6d8684d618a60cba6e523e0e1 P c22e9dc4838648d2bdaaa76af7ffc4b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "221d138c3adb48d6b239f506a8aaa778" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 39953 } } peers { permanent_uuid: "7d639c741543473587167bd497e3fc7d" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 37769 } } peers { permanent_uuid: "c22e9dc4838648d2bdaaa76af7ffc4b7" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 45991 } }
I20260925 02:43:05.531821 15599 tablet_replica.cc:439] T ee66e1f6d8684d618a60cba6e523e0e1 P c22e9dc4838648d2bdaaa76af7ffc4b7: TxnStatusTablet state changed. Reason: New leader c22e9dc4838648d2bdaaa76af7ffc4b7. Latest consensus state: current_term: 1 leader_uuid: "c22e9dc4838648d2bdaaa76af7ffc4b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "221d138c3adb48d6b239f506a8aaa778" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 39953 } } peers { permanent_uuid: "7d639c741543473587167bd497e3fc7d" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 37769 } } peers { permanent_uuid: "c22e9dc4838648d2bdaaa76af7ffc4b7" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 45991 } } }
I20260925 02:43:05.531919 15599 tablet_replica.cc:442] T ee66e1f6d8684d618a60cba6e523e0e1 P c22e9dc4838648d2bdaaa76af7ffc4b7: This TxnStatusTablet replica's current role is: LEADER
I20260925 02:43:05.532320 15625 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260925 02:43:05.532456 15625 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260925 02:43:05.532907 15061 catalog_manager.cc:5495] T ee66e1f6d8684d618a60cba6e523e0e1 P c22e9dc4838648d2bdaaa76af7ffc4b7 reported cstate change: term changed from 0 to 1, leader changed from <none> to c22e9dc4838648d2bdaaa76af7ffc4b7 (127.13.157.131). New cstate: current_term: 1 leader_uuid: "c22e9dc4838648d2bdaaa76af7ffc4b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "221d138c3adb48d6b239f506a8aaa778" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 39953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d639c741543473587167bd497e3fc7d" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 37769 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c22e9dc4838648d2bdaaa76af7ffc4b7" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 45991 } health_report { overall_health: HEALTHY } } }
I20260925 02:43:05.544395 15619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:05.540992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.131:45991 (local address 127.0.0.1:33694)
0925 02:43:05.541143 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:05.541160 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:05.541264 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:05.541520 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:05.541522 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:05.541528 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 02:43:05.541629 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:05.541634 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:05.542597 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:05.542600 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:05.543271 (+ 671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:05.543280 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:05.543803 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:05.543805 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:05.543869 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:05.543914 (+ 45us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 02:43:05.544172 (+ 258us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 02:43:05.544181 (+ 9us) client_negotiation.cc:769] Sending connection context
0925 02:43:05.544224 (+ 43us) client_negotiation.cc:240] Negotiation successful
0925 02:43:05.544290 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260925 02:43:05.544417 15579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:05.541093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33694 (local address 127.13.157.131:45991)
0925 02:43:05.541239 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:05.541244 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:05.541259 (+ 15us) server_negotiation.cc:407] Connection header received
0925 02:43:05.541368 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:05.541371 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:05.541419 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:05.541488 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 02:43:05.541713 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:05.542469 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:05.543363 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:05.543710 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:05.543808 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:05.544020 (+ 212us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 02:43:05.544107 (+ 87us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 02:43:05.544171 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:05.544236 (+ 65us) server_negotiation.cc:299] Negotiation successful
0925 02:43:05.544286 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260925 02:43:05.546561 15223 raft_consensus.cc:1261] T ee66e1f6d8684d618a60cba6e523e0e1 P 221d138c3adb48d6b239f506a8aaa778 [term 1 FOLLOWER]: Refusing update from remote peer c22e9dc4838648d2bdaaa76af7ffc4b7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:43:05.546654 15364 raft_consensus.cc:1261] T ee66e1f6d8684d618a60cba6e523e0e1 P 7d639c741543473587167bd497e3fc7d [term 1 FOLLOWER]: Refusing update from remote peer c22e9dc4838648d2bdaaa76af7ffc4b7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:43:05.547012 15591 consensus_queue.cc:1035] T ee66e1f6d8684d618a60cba6e523e0e1 P c22e9dc4838648d2bdaaa76af7ffc4b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d639c741543473587167bd497e3fc7d" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 37769 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:43:05.547161 15599 consensus_queue.cc:1035] T ee66e1f6d8684d618a60cba6e523e0e1 P c22e9dc4838648d2bdaaa76af7ffc4b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "221d138c3adb48d6b239f506a8aaa778" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 39953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:43:05.548403 15589 tablet_replica.cc:439] T ee66e1f6d8684d618a60cba6e523e0e1 P 221d138c3adb48d6b239f506a8aaa778: TxnStatusTablet state changed. Reason: New leader c22e9dc4838648d2bdaaa76af7ffc4b7. Latest consensus state: current_term: 1 leader_uuid: "c22e9dc4838648d2bdaaa76af7ffc4b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "221d138c3adb48d6b239f506a8aaa778" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 39953 } } peers { permanent_uuid: "7d639c741543473587167bd497e3fc7d" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 37769 } } peers { permanent_uuid: "c22e9dc4838648d2bdaaa76af7ffc4b7" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 45991 } } }
I20260925 02:43:05.548506 15589 tablet_replica.cc:442] T ee66e1f6d8684d618a60cba6e523e0e1 P 221d138c3adb48d6b239f506a8aaa778: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 02:43:05.548532 15587 tablet_replica.cc:439] T ee66e1f6d8684d618a60cba6e523e0e1 P 7d639c741543473587167bd497e3fc7d: TxnStatusTablet state changed. Reason: New leader c22e9dc4838648d2bdaaa76af7ffc4b7. Latest consensus state: current_term: 1 leader_uuid: "c22e9dc4838648d2bdaaa76af7ffc4b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "221d138c3adb48d6b239f506a8aaa778" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 39953 } } peers { permanent_uuid: "7d639c741543473587167bd497e3fc7d" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 37769 } } peers { permanent_uuid: "c22e9dc4838648d2bdaaa76af7ffc4b7" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 45991 } } }
I20260925 02:43:05.548633 15587 tablet_replica.cc:442] T ee66e1f6d8684d618a60cba6e523e0e1 P 7d639c741543473587167bd497e3fc7d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 02:43:05.550628 15588 tablet_replica.cc:439] T ee66e1f6d8684d618a60cba6e523e0e1 P c22e9dc4838648d2bdaaa76af7ffc4b7: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c22e9dc4838648d2bdaaa76af7ffc4b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "221d138c3adb48d6b239f506a8aaa778" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 39953 } } peers { permanent_uuid: "7d639c741543473587167bd497e3fc7d" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 37769 } } peers { permanent_uuid: "c22e9dc4838648d2bdaaa76af7ffc4b7" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 45991 } } }
I20260925 02:43:05.550731 15588 tablet_replica.cc:442] T ee66e1f6d8684d618a60cba6e523e0e1 P c22e9dc4838648d2bdaaa76af7ffc4b7: This TxnStatusTablet replica's current role is: LEADER
I20260925 02:43:05.550818 15594 mvcc.cc:205] Tried to move back new op lower bound from 7333085943994949632 to 7333085943937011712. Current Snapshot: MvccSnapshot[applied={T|T < 7333085943994949632}]
I20260925 02:43:05.551126 15587 tablet_replica.cc:439] T ee66e1f6d8684d618a60cba6e523e0e1 P 7d639c741543473587167bd497e3fc7d: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c22e9dc4838648d2bdaaa76af7ffc4b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "221d138c3adb48d6b239f506a8aaa778" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 39953 } } peers { permanent_uuid: "7d639c741543473587167bd497e3fc7d" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 37769 } } peers { permanent_uuid: "c22e9dc4838648d2bdaaa76af7ffc4b7" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 45991 } } }
I20260925 02:43:05.551219 15587 tablet_replica.cc:442] T ee66e1f6d8684d618a60cba6e523e0e1 P 7d639c741543473587167bd497e3fc7d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 02:43:05.551235 15599 tablet_replica.cc:439] T ee66e1f6d8684d618a60cba6e523e0e1 P c22e9dc4838648d2bdaaa76af7ffc4b7: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c22e9dc4838648d2bdaaa76af7ffc4b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "221d138c3adb48d6b239f506a8aaa778" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 39953 } } peers { permanent_uuid: "7d639c741543473587167bd497e3fc7d" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 37769 } } peers { permanent_uuid: "c22e9dc4838648d2bdaaa76af7ffc4b7" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 45991 } } }
I20260925 02:43:05.551340 15599 tablet_replica.cc:442] T ee66e1f6d8684d618a60cba6e523e0e1 P c22e9dc4838648d2bdaaa76af7ffc4b7: This TxnStatusTablet replica's current role is: LEADER
I20260925 02:43:05.552309 15589 tablet_replica.cc:439] T ee66e1f6d8684d618a60cba6e523e0e1 P 221d138c3adb48d6b239f506a8aaa778: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c22e9dc4838648d2bdaaa76af7ffc4b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "221d138c3adb48d6b239f506a8aaa778" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 39953 } } peers { permanent_uuid: "7d639c741543473587167bd497e3fc7d" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 37769 } } peers { permanent_uuid: "c22e9dc4838648d2bdaaa76af7ffc4b7" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 45991 } } }
I20260925 02:43:05.552408 15589 tablet_replica.cc:442] T ee66e1f6d8684d618a60cba6e523e0e1 P 221d138c3adb48d6b239f506a8aaa778: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 02:43:05.562577 15576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:05.558763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50998 (local address 127.13.157.130:37769)
0925 02:43:05.558882 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:05.558886 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:05.558900 (+ 14us) server_negotiation.cc:407] Connection header received
0925 02:43:05.559027 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:05.559030 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:05.559095 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:05.559202 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 02:43:05.559404 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:05.560188 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:05.560956 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:05.561296 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:05.561393 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:05.561598 (+ 205us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 02:43:05.562169 (+ 571us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 02:43:05.562272 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:05.562360 (+ 88us) server_negotiation.cc:299] Negotiation successful
0925 02:43:05.562412 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260925 02:43:05.575538 15632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:05.569670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.131:45991 (local address 127.13.157.130:37407)
0925 02:43:05.569936 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:05.569948 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:05.570030 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:05.570232 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:05.570235 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:05.570241 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 02:43:05.570387 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:05.570393 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:05.571382 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:05.571385 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:05.572081 (+ 696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:05.572091 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:05.574646 (+ 2555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:05.574653 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:05.574726 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:05.574875 (+ 149us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 02:43:05.575269 (+ 394us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 02:43:05.575275 (+ 6us) client_negotiation.cc:769] Sending connection context
0925 02:43:05.575333 (+ 58us) client_negotiation.cc:240] Negotiation successful
0925 02:43:05.575400 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":86,"threads_started":1}
I20260925 02:43:05.575556 15579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:05.569780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.130:37407 (local address 127.13.157.131:45991)
0925 02:43:05.569900 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:05.569904 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:05.570006 (+ 102us) server_negotiation.cc:407] Connection header received
0925 02:43:05.570098 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:05.570101 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:05.570151 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:05.570218 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 02:43:05.570521 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:05.571289 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:05.572186 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:05.572491 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:05.572599 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:05.575067 (+ 2468us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 02:43:05.575156 (+ 89us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 02:43:05.575239 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:05.575363 (+ 124us) server_negotiation.cc:299] Negotiation successful
0925 02:43:05.575419 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260925 02:43:05.581950 15223 raft_consensus.cc:1261] T f6fd9caa5f12412db461cdf94b7d04dc P 221d138c3adb48d6b239f506a8aaa778 [term 1 FOLLOWER]: Refusing update from remote peer 7d639c741543473587167bd497e3fc7d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:43:05.582023 15508 raft_consensus.cc:1261] T f6fd9caa5f12412db461cdf94b7d04dc P c22e9dc4838648d2bdaaa76af7ffc4b7 [term 1 FOLLOWER]: Refusing update from remote peer 7d639c741543473587167bd497e3fc7d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:43:05.582507 15612 consensus_queue.cc:1035] T f6fd9caa5f12412db461cdf94b7d04dc P 7d639c741543473587167bd497e3fc7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "221d138c3adb48d6b239f506a8aaa778" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 39953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:43:05.582706 15587 consensus_queue.cc:1035] T f6fd9caa5f12412db461cdf94b7d04dc P 7d639c741543473587167bd497e3fc7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c22e9dc4838648d2bdaaa76af7ffc4b7" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 45991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:43:05.605194 15637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:05.601417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.130:37769 (local address 127.13.157.131:41541)
0925 02:43:05.601761 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:05.601776 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:05.601893 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:05.602222 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:05.602225 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:05.602233 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 02:43:05.602450 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:05.602458 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:05.603310 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:05.603314 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:05.603977 (+ 663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:05.603988 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:05.604440 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:05.604444 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:05.604544 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:05.604593 (+ 49us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 02:43:05.604884 (+ 291us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 02:43:05.604892 (+ 8us) client_negotiation.cc:769] Sending connection context
0925 02:43:05.604958 (+ 66us) client_negotiation.cc:240] Negotiation successful
0925 02:43:05.605053 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":121,"threads_started":1}
I20260925 02:43:05.605171 15576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:05.601474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.131:41541 (local address 127.13.157.130:37769)
0925 02:43:05.601605 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:05.601609 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:05.601844 (+ 235us) server_negotiation.cc:407] Connection header received
0925 02:43:05.601989 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:05.601992 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:05.602041 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:05.602111 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 02:43:05.602567 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:05.603210 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:05.604095 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:05.604333 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:05.604431 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:05.604727 (+ 296us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 02:43:05.604805 (+ 78us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 02:43:05.604869 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:05.604967 (+ 98us) server_negotiation.cc:299] Negotiation successful
0925 02:43:05.605021 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260925 02:43:05.628334 15060 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40536:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 02:43:05.628507 15060 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40536:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 02:43:05.630858 15060 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ca875dfdc04b4faf9f1b8b2920cba8cf: Sending DeleteTablet for 3 replicas of tablet f6fd9caa5f12412db461cdf94b7d04dc
I20260925 02:43:05.631394 15202 tablet_service.cc:1514] Processing DeleteTablet for tablet f6fd9caa5f12412db461cdf94b7d04dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:43:05 UTC) from {username='kudu', principal='kudu/127.13.157.190@KRBTEST.COM'} at 127.0.0.1:38420
I20260925 02:43:05.631559 15604 tablet_replica.cc:330] T f6fd9caa5f12412db461cdf94b7d04dc P 221d138c3adb48d6b239f506a8aaa778: stopping tablet replica
I20260925 02:43:05.631664 15604 raft_consensus.cc:2229] T f6fd9caa5f12412db461cdf94b7d04dc P 221d138c3adb48d6b239f506a8aaa778 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 02:43:05.631759 15604 raft_consensus.cc:2258] T f6fd9caa5f12412db461cdf94b7d04dc P 221d138c3adb48d6b239f506a8aaa778 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:43:05.632004 15344 tablet_service.cc:1514] Processing DeleteTablet for tablet f6fd9caa5f12412db461cdf94b7d04dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:43:05 UTC) from {username='kudu', principal='kudu/127.13.157.190@KRBTEST.COM'} at 127.0.0.1:50988
I20260925 02:43:05.632197 15488 tablet_service.cc:1514] Processing DeleteTablet for tablet f6fd9caa5f12412db461cdf94b7d04dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:43:05 UTC) from {username='kudu', principal='kudu/127.13.157.190@KRBTEST.COM'} at 127.0.0.1:33686
I20260925 02:43:05.632246 15605 tablet_replica.cc:330] T f6fd9caa5f12412db461cdf94b7d04dc P 7d639c741543473587167bd497e3fc7d: stopping tablet replica
I20260925 02:43:05.632355 15606 tablet_replica.cc:330] T f6fd9caa5f12412db461cdf94b7d04dc P c22e9dc4838648d2bdaaa76af7ffc4b7: stopping tablet replica
I20260925 02:43:05.632349 15605 raft_consensus.cc:2229] T f6fd9caa5f12412db461cdf94b7d04dc P 7d639c741543473587167bd497e3fc7d [term 1 LEADER]: Raft consensus shutting down.
I20260925 02:43:05.632453 15606 raft_consensus.cc:2229] T f6fd9caa5f12412db461cdf94b7d04dc P c22e9dc4838648d2bdaaa76af7ffc4b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 02:43:05.632541 15606 raft_consensus.cc:2258] T f6fd9caa5f12412db461cdf94b7d04dc P c22e9dc4838648d2bdaaa76af7ffc4b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:43:05.632612 15605 raft_consensus.cc:2258] T f6fd9caa5f12412db461cdf94b7d04dc P 7d639c741543473587167bd497e3fc7d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:43:05.632759 15604 ts_tablet_manager.cc:1905] T f6fd9caa5f12412db461cdf94b7d04dc P 221d138c3adb48d6b239f506a8aaa778: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:43:05.633345 15605 ts_tablet_manager.cc:1905] T f6fd9caa5f12412db461cdf94b7d04dc P 7d639c741543473587167bd497e3fc7d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:43:05.636075 15605 ts_tablet_manager.cc:1918] T f6fd9caa5f12412db461cdf94b7d04dc P 7d639c741543473587167bd497e3fc7d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 02:43:05.636147 15605 log.cc:1196] T f6fd9caa5f12412db461cdf94b7d04dc P 7d639c741543473587167bd497e3fc7d: Deleting WAL directory at /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790304178325258-13942-0/minicluster-data/ts-1/wal/wals/f6fd9caa5f12412db461cdf94b7d04dc
I20260925 02:43:05.636423 15605 ts_tablet_manager.cc:1939] T f6fd9caa5f12412db461cdf94b7d04dc P 7d639c741543473587167bd497e3fc7d: Deleting consensus metadata
I20260925 02:43:05.637079 15049 catalog_manager.cc:4841] TS 7d639c741543473587167bd497e3fc7d (127.13.157.130:37769): tablet f6fd9caa5f12412db461cdf94b7d04dc (table test-table [id=35213401a781460e9821acee4d9646a1]) successfully deleted
I20260925 02:43:05.637135 15604 ts_tablet_manager.cc:1918] T f6fd9caa5f12412db461cdf94b7d04dc P 221d138c3adb48d6b239f506a8aaa778: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 02:43:05.637320 15604 log.cc:1196] T f6fd9caa5f12412db461cdf94b7d04dc P 221d138c3adb48d6b239f506a8aaa778: Deleting WAL directory at /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790304178325258-13942-0/minicluster-data/ts-0/wal/wals/f6fd9caa5f12412db461cdf94b7d04dc
I20260925 02:43:05.637810 15604 ts_tablet_manager.cc:1939] T f6fd9caa5f12412db461cdf94b7d04dc P 221d138c3adb48d6b239f506a8aaa778: Deleting consensus metadata
I20260925 02:43:05.637856 15606 ts_tablet_manager.cc:1905] T f6fd9caa5f12412db461cdf94b7d04dc P c22e9dc4838648d2bdaaa76af7ffc4b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:43:05.638567 15046 catalog_manager.cc:4841] TS 221d138c3adb48d6b239f506a8aaa778 (127.13.157.129:39953): tablet f6fd9caa5f12412db461cdf94b7d04dc (table test-table [id=35213401a781460e9821acee4d9646a1]) successfully deleted
I20260925 02:43:05.640506 15606 ts_tablet_manager.cc:1918] T f6fd9caa5f12412db461cdf94b7d04dc P c22e9dc4838648d2bdaaa76af7ffc4b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 02:43:05.640585 15606 log.cc:1196] T f6fd9caa5f12412db461cdf94b7d04dc P c22e9dc4838648d2bdaaa76af7ffc4b7: Deleting WAL directory at /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790304178325258-13942-0/minicluster-data/ts-2/wal/wals/f6fd9caa5f12412db461cdf94b7d04dc
I20260925 02:43:05.640832 15606 ts_tablet_manager.cc:1939] T f6fd9caa5f12412db461cdf94b7d04dc P c22e9dc4838648d2bdaaa76af7ffc4b7: Deleting consensus metadata
I20260925 02:43:05.641484 15049 catalog_manager.cc:4841] TS c22e9dc4838648d2bdaaa76af7ffc4b7 (127.13.157.131:45991): tablet f6fd9caa5f12412db461cdf94b7d04dc (table test-table [id=35213401a781460e9821acee4d9646a1]) successfully deleted
Sep 25 02:43:05 dist-test-slave-kznw krb5kdc[15017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304184, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.13.157.129@KRBTEST.COM
I20260925 02:43:05.651669 15577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:05.639209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38432 (local address 127.13.157.129:39953)
0925 02:43:05.639327 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:05.639332 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:05.639921 (+ 589us) server_negotiation.cc:407] Connection header received
0925 02:43:05.640132 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:05.640135 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:05.640198 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:05.640248 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:05.641402 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:05.642158 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:05.643429 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:05.643794 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:05.643894 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:05.647561 (+ 3667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:05.647576 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:05.647595 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:43:05.647624 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:05.649565 (+ 1941us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:05.650055 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:05.650062 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:05.650064 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:05.650192 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:05.650456 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:05.650459 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:05.650461 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:05.650781 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:05.651014 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:05.651212 (+ 198us) server_negotiation.cc:299] Negotiation successful
0925 02:43:05.651341 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
W20260925 02:43:05.652113 15256 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38432
I20260925 02:43:05.663703 15113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:05.654022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40556 (local address 127.13.157.190:32809)
0925 02:43:05.654146 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:05.654151 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:05.654577 (+ 426us) server_negotiation.cc:407] Connection header received
0925 02:43:05.654673 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:05.654682 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:05.654736 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:05.654817 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:05.655634 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:05.656461 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:05.657555 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:05.657920 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:05.658033 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:05.659802 (+ 1769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:05.659806 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:05.659809 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:43:05.659842 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:05.661577 (+ 1735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:05.662065 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:05.662069 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:05.662071 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:05.662162 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:05.662474 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:05.662477 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:05.662479 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:05.662714 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:05.662898 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:05.663393 (+ 495us) server_negotiation.cc:299] Negotiation successful
0925 02:43:05.663520 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
W20260925 02:43:05.664197 15060 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40556
I20260925 02:43:05.666301 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 15136
I20260925 02:43:05.675630 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 15279
I20260925 02:43:05.681912 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 15420
I20260925 02:43:05.688089 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 15030
2026-09-25T02:43:05Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2708 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790304178325258-13942-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 02:43:05 dist-test-slave-kznw krb5kdc[15646](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 02:43:05 dist-test-slave-kznw krb5kdc[15646](info): set up 2 sockets
Sep 25 02:43:05 dist-test-slave-kznw krb5kdc[15646](info): commencing operation
krb5kdc: starting...
W20260925 02:43:06.658185 13942 mini_kdc.cc:121] Time spent starting KDC: real 0.949s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790304178325258-13942-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790304178325258-13942-0/krb5kdc/test-user.keytab.
Sep 25 02:43:06 dist-test-slave-kznw krb5kdc[15646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304186, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T02:43:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T02:43:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.157.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.157.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.157.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab.
Entry for principal kudu/127.13.157.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab.
I20260925 02:43:06.778131 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790304178325258-13942-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790304178325258-13942-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790304178325258-13942-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.157.190:41363
--webserver_interface=127.13.157.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab
--principal=kudu/127.13.157.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:33829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.157.190:41363
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:43:06.886389 15659 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:43:06.886663 15659 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:43:06.886703 15659 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:43:06.890733 15659 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:43:06.890810 15659 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260925 02:43:06.890825 15659 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:43:06.890842 15659 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:43:06.890857 15659 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:43:06.890872 15659 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:43:06.896312 15659 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:33829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790304178325258-13942-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790304178325258-13942-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.157.190:41363
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab
--principal=kudu/127.13.157.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.157.190:41363
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.157.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15659
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790304178325258-13942-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:43:06.897388 15659 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:43:06.898408 15659 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 02:43:06.907014 15659 server_base.cc:1034] running on GCE node
W20260925 02:43:06.906952 15667 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:06.907032 15664 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:06.906952 15665 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:43:06.907735 15659 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:43:06.908824 15659 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:43:06.909986 15659 hybrid_clock.cc:648] HybridClock initialized: now 1790304186909945 us; error 52 us; skew 500 ppm
Sep 25 02:43:06 dist-test-slave-kznw krb5kdc[15646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304186, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:43:06.912652 15659 init.cc:377] Logged in from keytab as kudu/127.13.157.190@KRBTEST.COM (short username kudu)
I20260925 02:43:06.913537 15659 webserver.cc:460] Webserver started at http://127.13.157.190:41297/ using document root <none> and password file <none>
I20260925 02:43:06.914052 15659 fs_manager.cc:359] Metadata directory not provided
I20260925 02:43:06.914115 15659 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:43:06.914309 15659 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:43:06.916313 15659 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790304178325258-13942-0/minicluster-data/master-0/data/instance:
uuid: "fade460214df47e393ddd7d3df34fdfe"
format_stamp: "Formatted at 2026-09-25 02:43:06 on dist-test-slave-kznw"
server_key: "b84b1bf232f491da1919d432075662f2"
server_key_iv: "2a7c11456237f968b0994f8e45fddd59"
server_key_version: "encryptionkey@0"
I20260925 02:43:06.916839 15659 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790304178325258-13942-0/minicluster-data/master-0/wal/instance:
uuid: "fade460214df47e393ddd7d3df34fdfe"
format_stamp: "Formatted at 2026-09-25 02:43:06 on dist-test-slave-kznw"
server_key: "b84b1bf232f491da1919d432075662f2"
server_key_iv: "2a7c11456237f968b0994f8e45fddd59"
server_key_version: "encryptionkey@0"
I20260925 02:43:06.920449 15659 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260925 02:43:06.922799 15674 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:06.924058 15659 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260925 02:43:06.924170 15659 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790304178325258-13942-0/minicluster-data/master-0/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790304178325258-13942-0/minicluster-data/master-0/wal
uuid: "fade460214df47e393ddd7d3df34fdfe"
format_stamp: "Formatted at 2026-09-25 02:43:06 on dist-test-slave-kznw"
server_key: "b84b1bf232f491da1919d432075662f2"
server_key_iv: "2a7c11456237f968b0994f8e45fddd59"
server_key_version: "encryptionkey@0"
I20260925 02:43:06.924299 15659 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790304178325258-13942-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790304178325258-13942-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790304178325258-13942-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:43:06.944832 15659 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:43:06.947882 15659 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:43:06.948066 15659 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:43:06.955895 15659 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.190:41363
I20260925 02:43:06.955912 15737 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.190:41363 every 8 connection(s)
I20260925 02:43:06.956831 15659 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
I20260925 02:43:06.959643 15738 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:43:06.964174 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 15659
I20260925 02:43:06.964301 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790304178325258-13942-0/minicluster-data/master-0/wal/instance
I20260925 02:43:06.964584 13942 external_mini_cluster.cc:1428] Setting key 926131d818debbf03333fe182d7c48d8
I20260925 02:43:06.965590 15738 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fade460214df47e393ddd7d3df34fdfe: Bootstrap starting.
I20260925 02:43:06.968052 15738 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fade460214df47e393ddd7d3df34fdfe: Neither blocks nor log segments found. Creating new log.
I20260925 02:43:06.968942 15738 log.cc:824] T 00000000000000000000000000000000 P fade460214df47e393ddd7d3df34fdfe: Log is configured to *not* fsync() on all Append() calls
I20260925 02:43:06.971055 15738 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fade460214df47e393ddd7d3df34fdfe: No bootstrap required, opened a new log
Sep 25 02:43:06 dist-test-slave-kznw krb5kdc[15646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304186, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.157.190@KRBTEST.COM
I20260925 02:43:06.975328 15738 raft_consensus.cc:348] T 00000000000000000000000000000000 P fade460214df47e393ddd7d3df34fdfe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fade460214df47e393ddd7d3df34fdfe" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 41363 } }
I20260925 02:43:06.975553 15738 raft_consensus.cc:374] T 00000000000000000000000000000000 P fade460214df47e393ddd7d3df34fdfe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:43:06.975595 15738 raft_consensus.cc:729] T 00000000000000000000000000000000 P fade460214df47e393ddd7d3df34fdfe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fade460214df47e393ddd7d3df34fdfe, State: Initialized, Role: FOLLOWER
I20260925 02:43:06.976032 15738 consensus_queue.cc:260] T 00000000000000000000000000000000 P fade460214df47e393ddd7d3df34fdfe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fade460214df47e393ddd7d3df34fdfe" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 41363 } }
I20260925 02:43:06.976194 15738 raft_consensus.cc:388] T 00000000000000000000000000000000 P fade460214df47e393ddd7d3df34fdfe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 02:43:06.976263 15738 raft_consensus.cc:482] T 00000000000000000000000000000000 P fade460214df47e393ddd7d3df34fdfe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 02:43:06.976362 15738 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fade460214df47e393ddd7d3df34fdfe [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:43:06.977330 15738 raft_consensus.cc:504] T 00000000000000000000000000000000 P fade460214df47e393ddd7d3df34fdfe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fade460214df47e393ddd7d3df34fdfe" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 41363 } }
I20260925 02:43:06.977700 15738 leader_election.cc:302] T 00000000000000000000000000000000 P fade460214df47e393ddd7d3df34fdfe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fade460214df47e393ddd7d3df34fdfe; no voters:
I20260925 02:43:06.978060 15738 leader_election.cc:288] T 00000000000000000000000000000000 P fade460214df47e393ddd7d3df34fdfe [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 02:43:06.978173 15743 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fade460214df47e393ddd7d3df34fdfe [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:43:06.978401 15743 raft_consensus.cc:686] T 00000000000000000000000000000000 P fade460214df47e393ddd7d3df34fdfe [term 1 LEADER]: Becoming Leader. State: Replica: fade460214df47e393ddd7d3df34fdfe, State: Running, Role: LEADER
I20260925 02:43:06.978717 15743 consensus_queue.cc:237] T 00000000000000000000000000000000 P fade460214df47e393ddd7d3df34fdfe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fade460214df47e393ddd7d3df34fdfe" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 41363 } }
I20260925 02:43:06.979282 15738 sys_catalog.cc:564] T 00000000000000000000000000000000 P fade460214df47e393ddd7d3df34fdfe [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:43:06.979279 15741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:06.965999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41426 (local address 127.13.157.190:41363)
0925 02:43:06.966415 (+ 416us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:06.966431 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:06.966465 (+ 34us) server_negotiation.cc:407] Connection header received
0925 02:43:06.967219 (+ 754us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:06.967244 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:06.967579 (+ 335us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:06.967865 (+ 286us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:06.968722 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:06.969738 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:06.970829 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:06.971425 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:06.971554 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:06.974211 (+ 2657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:06.974216 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:06.974230 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:43:06.974260 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:06.975955 (+ 1695us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:06.976418 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:06.976424 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:06.976431 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:06.976562 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:06.976878 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:06.976881 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:06.976882 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:06.977306 (+ 424us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:06.977470 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:06.977785 (+ 315us) server_negotiation.cc:299] Negotiation successful
0925 02:43:06.978241 (+ 456us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":126,"threads_started":1}
I20260925 02:43:06.979979 15744 sys_catalog.cc:455] T 00000000000000000000000000000000 P fade460214df47e393ddd7d3df34fdfe [sys.catalog]: SysCatalogTable state changed. Reason: New leader fade460214df47e393ddd7d3df34fdfe. Latest consensus state: current_term: 1 leader_uuid: "fade460214df47e393ddd7d3df34fdfe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fade460214df47e393ddd7d3df34fdfe" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 41363 } } }
I20260925 02:43:06.980741 15744 sys_catalog.cc:458] T 00000000000000000000000000000000 P fade460214df47e393ddd7d3df34fdfe [sys.catalog]: This master's current role is: LEADER
I20260925 02:43:06.980633 15745 sys_catalog.cc:455] T 00000000000000000000000000000000 P fade460214df47e393ddd7d3df34fdfe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fade460214df47e393ddd7d3df34fdfe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fade460214df47e393ddd7d3df34fdfe" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 41363 } } }
I20260925 02:43:06.981063 15745 sys_catalog.cc:458] T 00000000000000000000000000000000 P fade460214df47e393ddd7d3df34fdfe [sys.catalog]: This master's current role is: LEADER
W20260925 02:43:06.984081 15758 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fade460214df47e393ddd7d3df34fdfe: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 02:43:06.984146 15758 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 02:43:06.984251 15753 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:43:06.985203 15753 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:43:06.991001 15753 catalog_manager.cc:1347] Generated new cluster ID: cee253b04f0645bfa513d1d9196b3dd6
I20260925 02:43:06.991070 15753 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 02:43:07.020535 15753 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 02:43:07.021924 15753 catalog_manager.cc:1504] Loading token signing keys...
I20260925 02:43:07.033556 15753 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fade460214df47e393ddd7d3df34fdfe: Generated new TSK 0
I20260925 02:43:07.034310 15753 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.157.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.157.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.157.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.129.keytab.
Entry for principal kudu/127.13.157.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.129.keytab.
I20260925 02:43:07.071799 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790304178325258-13942-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790304178325258-13942-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790304178325258-13942-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790304178325258-13942-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.157.129:0
--local_ip_for_outbound_sockets=127.13.157.129
--webserver_interface=127.13.157.129
--webserver_port=0
--tserver_master_addrs=127.13.157.190:41363
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.129.keytab
--principal=kudu/127.13.157.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:33829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:43:07.209290 15764 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:43:07.209559 15764 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:43:07.209615 15764 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:43:07.213559 15764 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:43:07.213621 15764 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:43:07.213660 15764 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260925 02:43:07.213672 15764 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 02:43:07.213702 15764 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.157.129
I20260925 02:43:07.218873 15764 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:33829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790304178325258-13942-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790304178325258-13942-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.129.keytab
--principal=kudu/127.13.157.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.157.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790304178325258-13942-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.157.129
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.157.190:41363
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15764
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.157.129
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790304178325258-13942-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:43:07.219903 15764 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:43:07.220873 15764 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:43:07.230357 15769 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:43:07.230468 15764 server_base.cc:1034] running on GCE node
W20260925 02:43:07.230371 15770 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:07.230371 15772 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:43:07.231041 15764 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:43:07.231688 15764 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:43:07.232857 15764 hybrid_clock.cc:648] HybridClock initialized: now 1790304187232837 us; error 32 us; skew 500 ppm
Sep 25 02:43:07 dist-test-slave-kznw krb5kdc[15646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304187, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:43:07.235548 15764 init.cc:377] Logged in from keytab as kudu/127.13.157.129@KRBTEST.COM (short username kudu)
I20260925 02:43:07.236404 15764 webserver.cc:460] Webserver started at http://127.13.157.129:36991/ using document root <none> and password file <none>
I20260925 02:43:07.236920 15764 fs_manager.cc:359] Metadata directory not provided
I20260925 02:43:07.236979 15764 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:43:07.237159 15764 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:43:07.239279 15764 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790304178325258-13942-0/minicluster-data/ts-0/data/instance:
uuid: "bf02cb03155045689c3075ab6290276d"
format_stamp: "Formatted at 2026-09-25 02:43:07 on dist-test-slave-kznw"
server_key: "5752b53c9a3205209b06edf56bc0e261"
server_key_iv: "a2bd37f00c95a6e2dfe07206a9a360a6"
server_key_version: "encryptionkey@0"
I20260925 02:43:07.239936 15764 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790304178325258-13942-0/minicluster-data/ts-0/wal/instance:
uuid: "bf02cb03155045689c3075ab6290276d"
format_stamp: "Formatted at 2026-09-25 02:43:07 on dist-test-slave-kznw"
server_key: "5752b53c9a3205209b06edf56bc0e261"
server_key_iv: "a2bd37f00c95a6e2dfe07206a9a360a6"
server_key_version: "encryptionkey@0"
I20260925 02:43:07.243973 15764 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260925 02:43:07.246601 15779 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:07.248023 15764 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 02:43:07.248145 15764 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790304178325258-13942-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790304178325258-13942-0/minicluster-data/ts-0/wal
uuid: "bf02cb03155045689c3075ab6290276d"
format_stamp: "Formatted at 2026-09-25 02:43:07 on dist-test-slave-kznw"
server_key: "5752b53c9a3205209b06edf56bc0e261"
server_key_iv: "a2bd37f00c95a6e2dfe07206a9a360a6"
server_key_version: "encryptionkey@0"
I20260925 02:43:07.248263 15764 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790304178325258-13942-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790304178325258-13942-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790304178325258-13942-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:43:07.267132 15764 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:43:07.270066 15764 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:43:07.270215 15764 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:43:07.271688 15764 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:43:07.271741 15764 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:07.271782 15764 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:43:07.271811 15764 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 25 02:43:07 dist-test-slave-kznw krb5kdc[15646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304187, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.129@KRBTEST.COM for kudu/127.13.157.190@KRBTEST.COM
I20260925 02:43:07.281867 15764 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.129:41557
I20260925 02:43:07.282745 15764 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790304178325258-13942-0/minicluster-data/ts-0/data/info.pb
I20260925 02:43:07.283041 15899 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.129:41557 every 8 connection(s)
I20260925 02:43:07.284647 15741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:07.272441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.129:45843 (local address 127.13.157.190:41363)
0925 02:43:07.272600 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:07.272606 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:07.273437 (+ 831us) server_negotiation.cc:407] Connection header received
0925 02:43:07.274504 (+ 1067us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:07.274508 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:07.274567 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:07.274655 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:07.276049 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:07.276570 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:07.277781 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:07.278076 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:07.278173 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:07.281089 (+ 2916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:07.281092 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:07.281095 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:43:07.281123 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:07.282608 (+ 1485us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:07.283193 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:07.283196 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:07.283198 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:07.283289 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:07.283666 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:07.283669 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:07.283671 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:07.283900 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:07.283986 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:07.284412 (+ 426us) server_negotiation.cc:299] Negotiation successful
0925 02:43:07.284529 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260925 02:43:07.285367 15789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:07.272813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:41363 (local address 127.13.157.129:45843)
0925 02:43:07.273273 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:07.273323 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:07.274243 (+ 920us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:07.274867 (+ 624us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:07.274876 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:07.275291 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:07.275891 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:07.275911 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:07.276689 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:07.276693 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:07.277665 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:07.277675 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:07.278241 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:07.278245 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:07.278400 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:07.278986 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:07.279005 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:07.280927 (+ 1922us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:43:07.282722 (+ 1795us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:07.282728 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:07.282752 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:07.283065 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:07.283393 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:07.283396 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:07.283398 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:07.283557 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:07.283989 (+ 432us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:43:07.283994 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:43:07.284212 (+ 218us) client_negotiation.cc:769] Sending connection context
0925 02:43:07.284353 (+ 141us) client_negotiation.cc:240] Negotiation successful
0925 02:43:07.284580 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":117,"threads_started":1}
I20260925 02:43:07.286638 15901 heartbeater.cc:347] Connected to a master server at 127.13.157.190:41363
I20260925 02:43:07.286787 15901 heartbeater.cc:464] Registering TS with master...
I20260925 02:43:07.287267 15901 heartbeater.cc:511] Master 127.13.157.190:41363 requested a full tablet report, sending...
I20260925 02:43:07.288714 15691 ts_manager.cc:194] Registered new tserver with Master: bf02cb03155045689c3075ab6290276d (127.13.157.129:41557)
I20260925 02:43:07.289132 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 15764
I20260925 02:43:07.289289 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790304178325258-13942-0/minicluster-data/ts-0/wal/instance
I20260925 02:43:07.289539 13942 external_mini_cluster.cc:1428] Setting key 7d789f16b0182f0ab12cc7df41eac84b
I20260925 02:43:07.289968 15691 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.157.129@KRBTEST.COM'} at 127.13.157.129:45843
I20260925 02:43:07.297096 15741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:07.288657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.129:59101 (local address 127.13.157.190:41363)
0925 02:43:07.290591 (+ 1934us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:07.290596 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:07.290609 (+ 13us) server_negotiation.cc:407] Connection header received
0925 02:43:07.290642 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:07.290645 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:07.290683 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:07.290764 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:07.291327 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:07.292054 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:07.292953 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:07.293175 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:07.293261 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:07.294251 (+ 990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:07.294254 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:07.294255 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:43:07.294277 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:07.295590 (+ 1313us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:07.295982 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:07.295985 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:07.295987 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:07.296060 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:07.296311 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:07.296314 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:07.296316 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:07.296487 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:07.296571 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:07.296826 (+ 255us) server_negotiation.cc:299] Negotiation successful
0925 02:43:07.296935 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1866}
I20260925 02:43:07.297107 15907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:07.288558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:41363 (local address 127.13.157.129:59101)
0925 02:43:07.288838 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:07.288852 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:07.288936 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:07.290803 (+ 1867us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:07.290806 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:07.291061 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:07.291198 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:07.291206 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:07.292195 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:07.292198 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:07.292841 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:07.292851 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:07.293274 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:07.293278 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:07.293368 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:07.293816 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:07.293829 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:07.294123 (+ 294us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:43:07.295698 (+ 1575us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:07.295701 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:07.295704 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:07.295895 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:07.296141 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:07.296144 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:07.296146 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:07.296227 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:07.296577 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:43:07.296580 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:43:07.296677 (+ 97us) client_negotiation.cc:769] Sending connection context
0925 02:43:07.296761 (+ 84us) client_negotiation.cc:240] Negotiation successful
0925 02:43:07.296915 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"mutex_wait_us":2,"thread_start_us":91,"threads_started":1}
WARNING: no policy specified for kudu/127.13.157.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.157.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.157.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.130.keytab.
Entry for principal kudu/127.13.157.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.130.keytab.
I20260925 02:43:07.316213 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790304178325258-13942-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790304178325258-13942-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790304178325258-13942-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790304178325258-13942-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.157.130:0
--local_ip_for_outbound_sockets=127.13.157.130
--webserver_interface=127.13.157.130
--webserver_port=0
--tserver_master_addrs=127.13.157.190:41363
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.130.keytab
--principal=kudu/127.13.157.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:33829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:43:07.424892 15910 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:43:07.425166 15910 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:43:07.425243 15910 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:43:07.429256 15910 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:43:07.429435 15910 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:43:07.429538 15910 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260925 02:43:07.429566 15910 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 02:43:07.429633 15910 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.157.130
I20260925 02:43:07.435319 15910 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:33829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790304178325258-13942-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790304178325258-13942-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.130.keytab
--principal=kudu/127.13.157.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.157.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790304178325258-13942-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.157.130
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.157.190:41363
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15910
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.157.130
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790304178325258-13942-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:43:07.436395 15910 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:43:07.437434 15910 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 02:43:07.447063 15910 server_base.cc:1034] running on GCE node
W20260925 02:43:07.446998 15915 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:07.447019 15916 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:07.447019 15918 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:43:07.447584 15910 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:43:07.448218 15910 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:43:07.449384 15910 hybrid_clock.cc:648] HybridClock initialized: now 1790304187449362 us; error 34 us; skew 500 ppm
Sep 25 02:43:07 dist-test-slave-kznw krb5kdc[15646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304187, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:43:07.452140 15910 init.cc:377] Logged in from keytab as kudu/127.13.157.130@KRBTEST.COM (short username kudu)
I20260925 02:43:07.453016 15910 webserver.cc:460] Webserver started at http://127.13.157.130:41977/ using document root <none> and password file <none>
I20260925 02:43:07.453516 15910 fs_manager.cc:359] Metadata directory not provided
I20260925 02:43:07.453572 15910 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:43:07.453737 15910 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:43:07.455718 15910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790304178325258-13942-0/minicluster-data/ts-1/data/instance:
uuid: "aa9d93662a1a4a5dbb0bd66f472479eb"
format_stamp: "Formatted at 2026-09-25 02:43:07 on dist-test-slave-kznw"
server_key: "0eea4513ea2923c75d4c88b4f0394bdb"
server_key_iv: "b3fe217fc286ca30003e5aa86bb7d292"
server_key_version: "encryptionkey@0"
I20260925 02:43:07.456235 15910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790304178325258-13942-0/minicluster-data/ts-1/wal/instance:
uuid: "aa9d93662a1a4a5dbb0bd66f472479eb"
format_stamp: "Formatted at 2026-09-25 02:43:07 on dist-test-slave-kznw"
server_key: "0eea4513ea2923c75d4c88b4f0394bdb"
server_key_iv: "b3fe217fc286ca30003e5aa86bb7d292"
server_key_version: "encryptionkey@0"
I20260925 02:43:07.459800 15910 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260925 02:43:07.462231 15926 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:07.463533 15910 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260925 02:43:07.463634 15910 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790304178325258-13942-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790304178325258-13942-0/minicluster-data/ts-1/wal
uuid: "aa9d93662a1a4a5dbb0bd66f472479eb"
format_stamp: "Formatted at 2026-09-25 02:43:07 on dist-test-slave-kznw"
server_key: "0eea4513ea2923c75d4c88b4f0394bdb"
server_key_iv: "b3fe217fc286ca30003e5aa86bb7d292"
server_key_version: "encryptionkey@0"
I20260925 02:43:07.463753 15910 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790304178325258-13942-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790304178325258-13942-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790304178325258-13942-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:43:07.500253 15910 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:43:07.503479 15910 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:43:07.503659 15910 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:43:07.505108 15910 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:43:07.505156 15910 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:07.505209 15910 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:43:07.505239 15910 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 25 02:43:07 dist-test-slave-kznw krb5kdc[15646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304187, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.130@KRBTEST.COM for kudu/127.13.157.190@KRBTEST.COM
I20260925 02:43:07.515690 15910 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.130:33685
I20260925 02:43:07.515724 16045 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.130:33685 every 8 connection(s)
I20260925 02:43:07.516588 15910 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790304178325258-13942-0/minicluster-data/ts-1/data/info.pb
I20260925 02:43:07.519126 15741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:07.505992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.130:58073 (local address 127.13.157.190:41363)
0925 02:43:07.506104 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:07.506108 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:07.507335 (+ 1227us) server_negotiation.cc:407] Connection header received
0925 02:43:07.508486 (+ 1151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:07.508489 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:07.508554 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:07.508639 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:07.510049 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:07.510709 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:07.511987 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:07.512230 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:07.512364 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:07.515519 (+ 3155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:07.515523 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:07.515526 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:43:07.515558 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:07.516978 (+ 1420us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:07.517591 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:07.517595 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:07.517597 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:07.517692 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:07.518024 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:07.518027 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:07.518029 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:07.518225 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:07.518311 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:07.518812 (+ 501us) server_negotiation.cc:299] Negotiation successful
0925 02:43:07.518961 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260925 02:43:07.519923 15936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:07.506587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:41363 (local address 127.13.157.130:58073)
0925 02:43:07.507172 (+ 585us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:07.507218 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:07.508241 (+ 1023us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:07.508845 (+ 604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:07.508855 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:07.509225 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:07.509868 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:07.509885 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:07.510839 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:07.510843 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:07.511861 (+ 1018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:07.511872 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:07.512464 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:07.512470 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:07.512683 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:07.513358 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:07.513382 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:07.515311 (+ 1929us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:43:07.517121 (+ 1810us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:07.517126 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:07.517152 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:07.517481 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:07.517778 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:07.517781 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:07.517783 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:07.517926 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:07.518327 (+ 401us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:43:07.518334 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:43:07.518572 (+ 238us) client_negotiation.cc:769] Sending connection context
0925 02:43:07.518735 (+ 163us) client_negotiation.cc:240] Negotiation successful
0925 02:43:07.519087 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":144,"threads_started":1}
I20260925 02:43:07.521144 16046 heartbeater.cc:347] Connected to a master server at 127.13.157.190:41363
I20260925 02:43:07.521303 16046 heartbeater.cc:464] Registering TS with master...
I20260925 02:43:07.521880 16046 heartbeater.cc:511] Master 127.13.157.190:41363 requested a full tablet report, sending...
I20260925 02:43:07.522704 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 15910
I20260925 02:43:07.522816 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790304178325258-13942-0/minicluster-data/ts-1/wal/instance
I20260925 02:43:07.523164 13942 external_mini_cluster.cc:1428] Setting key 24c06f39c00309ed7766a29eda1361f1
I20260925 02:43:07.523144 15691 ts_manager.cc:194] Registered new tserver with Master: aa9d93662a1a4a5dbb0bd66f472479eb (127.13.157.130:33685)
I20260925 02:43:07.523867 15691 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.157.130@KRBTEST.COM'} at 127.13.157.130:58073
I20260925 02:43:07.531003 15741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:07.523513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.130:43713 (local address 127.13.157.190:41363)
0925 02:43:07.523719 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:07.523724 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:07.523855 (+ 131us) server_negotiation.cc:407] Connection header received
0925 02:43:07.523956 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:07.523959 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:07.524002 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:07.524067 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:07.524952 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:07.525696 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:07.526531 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:07.526855 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:07.526987 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:07.528056 (+ 1069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:07.528060 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:07.528062 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:43:07.528088 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:07.529465 (+ 1377us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:07.529852 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:07.529856 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:07.529858 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:07.529934 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:07.530174 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:07.530177 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:07.530179 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:07.530388 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:07.530475 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:07.530720 (+ 245us) server_negotiation.cc:299] Negotiation successful
0925 02:43:07.530836 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124}
I20260925 02:43:07.531014 16052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:07.523455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:41363 (local address 127.13.157.130:43713)
0925 02:43:07.523778 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:07.523792 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:07.523883 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:07.524488 (+ 605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:07.524491 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:07.524703 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:07.524821 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:07.524828 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:07.525831 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:07.525834 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:07.526417 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:07.526426 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:07.526996 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:07.526999 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:07.527070 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:07.527570 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:07.527582 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:07.527931 (+ 349us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:43:07.529567 (+ 1636us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:07.529571 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:07.529573 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:07.529766 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:07.530014 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:07.530017 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:07.530019 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:07.530098 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:07.530480 (+ 382us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:43:07.530484 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:43:07.530571 (+ 87us) client_negotiation.cc:769] Sending connection context
0925 02:43:07.530654 (+ 83us) client_negotiation.cc:240] Negotiation successful
0925 02:43:07.530793 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":86,"threads_started":1}
WARNING: no policy specified for kudu/127.13.157.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.157.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.157.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.131.keytab.
Entry for principal kudu/127.13.157.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.131.keytab.
I20260925 02:43:07.549348 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790304178325258-13942-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790304178325258-13942-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790304178325258-13942-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790304178325258-13942-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.157.131:0
--local_ip_for_outbound_sockets=127.13.157.131
--webserver_interface=127.13.157.131
--webserver_port=0
--tserver_master_addrs=127.13.157.190:41363
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.131.keytab
--principal=kudu/127.13.157.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:33829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:43:07.661142 16055 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:43:07.661401 16055 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:43:07.661448 16055 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:43:07.665546 16055 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:43:07.665608 16055 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:43:07.665648 16055 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260925 02:43:07.665663 16055 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 02:43:07.665715 16055 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.157.131
I20260925 02:43:07.671188 16055 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:33829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790304178325258-13942-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790304178325258-13942-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.131.keytab
--principal=kudu/127.13.157.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.157.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790304178325258-13942-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.157.131
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.157.190:41363
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16055
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.157.131
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790304178325258-13942-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:43:07.672286 16055 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:43:07.673310 16055 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:43:07.682938 16060 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:07.682960 16061 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:07.683007 16063 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:43:07.683096 16055 server_base.cc:1034] running on GCE node
I20260925 02:43:07.683578 16055 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:43:07.684229 16055 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:43:07.685401 16055 hybrid_clock.cc:648] HybridClock initialized: now 1790304187685381 us; error 31 us; skew 500 ppm
Sep 25 02:43:07 dist-test-slave-kznw krb5kdc[15646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304187, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:43:07.688086 16055 init.cc:377] Logged in from keytab as kudu/127.13.157.131@KRBTEST.COM (short username kudu)
I20260925 02:43:07.689013 16055 webserver.cc:460] Webserver started at http://127.13.157.131:38219/ using document root <none> and password file <none>
I20260925 02:43:07.689548 16055 fs_manager.cc:359] Metadata directory not provided
I20260925 02:43:07.689615 16055 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:43:07.689819 16055 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:43:07.691870 16055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790304178325258-13942-0/minicluster-data/ts-2/data/instance:
uuid: "7e2bc888fce84507ba5be9f8efdbf554"
format_stamp: "Formatted at 2026-09-25 02:43:07 on dist-test-slave-kznw"
server_key: "0159dd788c2b97b45e834c84c38586a4"
server_key_iv: "50dc0e60d76e7c116254a64335f9fdb8"
server_key_version: "encryptionkey@0"
I20260925 02:43:07.692420 16055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790304178325258-13942-0/minicluster-data/ts-2/wal/instance:
uuid: "7e2bc888fce84507ba5be9f8efdbf554"
format_stamp: "Formatted at 2026-09-25 02:43:07 on dist-test-slave-kznw"
server_key: "0159dd788c2b97b45e834c84c38586a4"
server_key_iv: "50dc0e60d76e7c116254a64335f9fdb8"
server_key_version: "encryptionkey@0"
I20260925 02:43:07.696101 16055 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.003s
I20260925 02:43:07.698585 16070 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:07.699980 16055 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260925 02:43:07.700088 16055 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790304178325258-13942-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790304178325258-13942-0/minicluster-data/ts-2/wal
uuid: "7e2bc888fce84507ba5be9f8efdbf554"
format_stamp: "Formatted at 2026-09-25 02:43:07 on dist-test-slave-kznw"
server_key: "0159dd788c2b97b45e834c84c38586a4"
server_key_iv: "50dc0e60d76e7c116254a64335f9fdb8"
server_key_version: "encryptionkey@0"
I20260925 02:43:07.700194 16055 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790304178325258-13942-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790304178325258-13942-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790304178325258-13942-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:43:07.714640 16055 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:43:07.717370 16055 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:43:07.717546 16055 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:43:07.719089 16055 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:43:07.719138 16055 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:07.719193 16055 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:43:07.719218 16055 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 25 02:43:07 dist-test-slave-kznw krb5kdc[15646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304187, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.131@KRBTEST.COM for kudu/127.13.157.190@KRBTEST.COM
I20260925 02:43:07.728978 16055 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.131:33691
I20260925 02:43:07.729031 16188 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.131:33691 every 8 connection(s)
I20260925 02:43:07.730042 16055 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790304178325258-13942-0/minicluster-data/ts-2/data/info.pb
I20260925 02:43:07.732131 15741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:07.719681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.131:59377 (local address 127.13.157.190:41363)
0925 02:43:07.719846 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:07.719852 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:07.721141 (+ 1289us) server_negotiation.cc:407] Connection header received
0925 02:43:07.722272 (+ 1131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:07.722275 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:07.722332 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:07.722407 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:07.723852 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:07.724401 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:07.725662 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:07.726010 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:07.726130 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:07.728677 (+ 2547us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:07.728681 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:07.728684 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:43:07.728716 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:07.730224 (+ 1508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:07.730770 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:07.730773 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:07.730775 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:07.730860 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:07.731191 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:07.731193 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:07.731194 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:07.731359 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:07.731432 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:07.731890 (+ 458us) server_negotiation.cc:299] Negotiation successful
0925 02:43:07.732006 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260925 02:43:07.732820 16079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:07.720530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:41363 (local address 127.13.157.131:59377)
0925 02:43:07.720999 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:07.721047 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:07.722072 (+ 1025us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:07.722599 (+ 527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:07.722605 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:07.723059 (+ 454us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:07.723717 (+ 658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:07.723736 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:07.724516 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:07.724520 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:07.725528 (+ 1008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:07.725539 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:07.726155 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:07.726158 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:07.726297 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:07.726814 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:07.726829 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:07.728498 (+ 1669us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:43:07.730325 (+ 1827us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:07.730329 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:07.730340 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:07.730643 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:07.730964 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:07.730968 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:07.730970 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:07.731114 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:07.731431 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:43:07.731436 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:43:07.731665 (+ 229us) client_negotiation.cc:769] Sending connection context
0925 02:43:07.731817 (+ 152us) client_negotiation.cc:240] Negotiation successful
0925 02:43:07.732047 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":129,"threads_started":1}
I20260925 02:43:07.733929 16189 heartbeater.cc:347] Connected to a master server at 127.13.157.190:41363
I20260925 02:43:07.734059 16189 heartbeater.cc:464] Registering TS with master...
I20260925 02:43:07.734588 16189 heartbeater.cc:511] Master 127.13.157.190:41363 requested a full tablet report, sending...
I20260925 02:43:07.735565 15691 ts_manager.cc:194] Registered new tserver with Master: 7e2bc888fce84507ba5be9f8efdbf554 (127.13.157.131:33691)
I20260925 02:43:07.735651 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 16055
I20260925 02:43:07.735785 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790304178325258-13942-0/minicluster-data/ts-2/wal/instance
I20260925 02:43:07.736130 13942 external_mini_cluster.cc:1428] Setting key 2b73f752a601bd9e74a966aee9afac8e
I20260925 02:43:07.736256 15691 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.157.131@KRBTEST.COM'} at 127.13.157.131:59377
I20260925 02:43:07.738085 13942 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 02:43:07.743572 15741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:07.735954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.131:37703 (local address 127.13.157.190:41363)
0925 02:43:07.736101 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:07.736105 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:07.736264 (+ 159us) server_negotiation.cc:407] Connection header received
0925 02:43:07.736416 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:07.736419 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:07.736461 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:07.736539 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:07.737319 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:07.738029 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:07.739101 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:07.739315 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:07.739391 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:07.740421 (+ 1030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:07.740425 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:07.740427 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:43:07.740453 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:07.741889 (+ 1436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:07.742268 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:07.742272 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:07.742274 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:07.742357 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:07.742738 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:07.742741 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:07.742743 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:07.742970 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:07.743069 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:07.743365 (+ 296us) server_negotiation.cc:299] Negotiation successful
0925 02:43:07.743460 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260925 02:43:07.743661 16195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:07.735885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:41363 (local address 127.13.157.131:37703)
0925 02:43:07.736186 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:07.736200 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:07.736300 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:07.736858 (+ 558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:07.736861 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:07.737069 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:07.737194 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:07.737202 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:07.738157 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:07.738160 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:07.738996 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:07.739006 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:07.739409 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:07.739413 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:07.739498 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:07.739939 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:07.739952 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:07.740293 (+ 341us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:43:07.741990 (+ 1697us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:07.741993 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:07.741994 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:07.742177 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:07.742507 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:07.742511 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:07.742513 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:07.742632 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:07.743091 (+ 459us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:43:07.743095 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:43:07.743184 (+ 89us) client_negotiation.cc:769] Sending connection context
0925 02:43:07.743297 (+ 113us) client_negotiation.cc:240] Negotiation successful
0925 02:43:07.743443 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":87,"threads_started":1}
Sep 25 02:43:07 dist-test-slave-kznw krb5kdc[15646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304187, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 25 02:43:07 dist-test-slave-kznw krb5kdc[15646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304187, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.157.190@KRBTEST.COM
I20260925 02:43:07.764593 15741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:07.756922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41438 (local address 127.13.157.190:41363)
0925 02:43:07.757055 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:07.757058 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:07.757071 (+ 13us) server_negotiation.cc:407] Connection header received
0925 02:43:07.757111 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:07.757113 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:07.757188 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:07.757262 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:07.757874 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:07.758411 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:07.759249 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:07.759463 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:07.759543 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:07.761747 (+ 2204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:07.761750 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:07.761752 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:43:07.761777 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:07.763286 (+ 1509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:07.763642 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:07.763645 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:07.763646 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:07.763722 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:07.763909 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:07.763911 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:07.763913 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:07.764080 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:07.764184 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:07.764370 (+ 186us) server_negotiation.cc:299] Negotiation successful
0925 02:43:07.764468 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260925 02:43:07.767642 15691 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41438:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 02:43:07.770782 15691 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 02:43:07.785548 16211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:07.780047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40784 (local address 127.13.157.131:33691)
0925 02:43:07.780736 (+ 689us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:07.780746 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:07.780765 (+ 19us) server_negotiation.cc:407] Connection header received
0925 02:43:07.780828 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:07.780832 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:07.780980 (+ 148us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:07.781421 (+ 441us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:43:07.781543 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:07.782618 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:07.783912 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:07.784701 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:07.784859 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:07.784991 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:07.785283 (+ 292us) server_negotiation.cc:299] Negotiation successful
0925 02:43:07.785380 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":606,"thread_start_us":118,"threads_started":1}
I20260925 02:43:07.785569 16208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:07.779864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.129:41557 (local address 127.0.0.1:53550)
0925 02:43:07.780531 (+ 667us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:07.780544 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:07.780724 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:07.781377 (+ 653us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:07.781380 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:07.781396 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:43:07.781479 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:07.781483 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:07.782432 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:07.782435 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:07.783877 (+ 1442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:07.783889 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:07.784848 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:07.784852 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:07.785062 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:07.785156 (+ 94us) client_negotiation.cc:769] Sending connection context
0925 02:43:07.785242 (+ 86us) client_negotiation.cc:240] Negotiation successful
0925 02:43:07.785404 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":608,"mutex_wait_us":101,"thread_start_us":91,"threads_started":1}
I20260925 02:43:07.786043 16210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:07.780020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53550 (local address 127.13.157.129:41557)
0925 02:43:07.780560 (+ 540us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:07.780573 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:07.780825 (+ 252us) server_negotiation.cc:407] Connection header received
0925 02:43:07.780895 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:07.780899 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:07.781071 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:07.781224 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:43:07.781571 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:07.782322 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:07.783988 (+ 1666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:07.784734 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:07.784858 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:07.784992 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:07.785801 (+ 809us) server_negotiation.cc:299] Negotiation successful
0925 02:43:07.785905 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":450,"thread_start_us":72,"threads_started":1}
I20260925 02:43:07.785569 16206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:07.779137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.131:33691 (local address 127.0.0.1:40784)
0925 02:43:07.779466 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:07.779513 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:07.779634 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:07.781114 (+ 1480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:07.781117 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:07.781134 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:43:07.781245 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:07.781250 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:07.782698 (+ 1448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:07.782702 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:07.783766 (+ 1064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:07.783776 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:07.784848 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:07.784852 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:07.785061 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:07.785128 (+ 67us) client_negotiation.cc:769] Sending connection context
0925 02:43:07.785242 (+ 114us) client_negotiation.cc:240] Negotiation successful
0925 02:43:07.785404 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"mutex_wait_us":69,"thread_start_us":75,"threads_started":1}
I20260925 02:43:07.786672 16207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:07.779245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.130:33685 (local address 127.0.0.1:57830)
0925 02:43:07.780189 (+ 944us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:07.780201 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:07.780341 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:07.781044 (+ 703us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:07.781049 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:07.781071 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:43:07.781229 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:07.781237 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:07.782643 (+ 1406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:07.782646 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:07.783889 (+ 1243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:07.783896 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:07.785914 (+ 2018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:07.785917 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:07.786038 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:07.786062 (+ 24us) client_negotiation.cc:769] Sending connection context
0925 02:43:07.786113 (+ 51us) client_negotiation.cc:240] Negotiation successful
0925 02:43:07.786323 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":884,"mutex_wait_us":72,"thread_start_us":105,"threads_started":1}
I20260925 02:43:07.789063 15831 tablet_service.cc:1467] Processing CreateTablet for tablet 400541e581fe412a94781ad06cd21167 (DEFAULT_TABLE table=test-table [id=65191a6cf8eb43d2bcabca6cfdcb003c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:43:07.789064 16120 tablet_service.cc:1467] Processing CreateTablet for tablet 400541e581fe412a94781ad06cd21167 (DEFAULT_TABLE table=test-table [id=65191a6cf8eb43d2bcabca6cfdcb003c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:43:07.790213 15831 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 400541e581fe412a94781ad06cd21167. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:43:07.790217 16120 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 400541e581fe412a94781ad06cd21167. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:43:07.791208 16209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:07.779937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57830 (local address 127.13.157.130:33685)
0925 02:43:07.780529 (+ 592us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:07.780545 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:07.780569 (+ 24us) server_negotiation.cc:407] Connection header received
0925 02:43:07.780646 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:07.780651 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:07.780890 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:07.781050 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:43:07.781344 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:07.782520 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:07.785050 (+ 2530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:07.785785 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:07.786865 (+ 1080us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:07.787028 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:07.787091 (+ 63us) server_negotiation.cc:299] Negotiation successful
0925 02:43:07.791063 (+ 3972us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":493,"thread_start_us":141,"threads_started":1}
I20260925 02:43:07.793416 15977 tablet_service.cc:1467] Processing CreateTablet for tablet 400541e581fe412a94781ad06cd21167 (DEFAULT_TABLE table=test-table [id=65191a6cf8eb43d2bcabca6cfdcb003c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:43:07.794276 15977 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 400541e581fe412a94781ad06cd21167. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:43:07.797744 16212 tablet_bootstrap.cc:492] T 400541e581fe412a94781ad06cd21167 P bf02cb03155045689c3075ab6290276d: Bootstrap starting.
I20260925 02:43:07.798506 16213 tablet_bootstrap.cc:492] T 400541e581fe412a94781ad06cd21167 P 7e2bc888fce84507ba5be9f8efdbf554: Bootstrap starting.
I20260925 02:43:07.800298 16213 tablet_bootstrap.cc:654] T 400541e581fe412a94781ad06cd21167 P 7e2bc888fce84507ba5be9f8efdbf554: Neither blocks nor log segments found. Creating new log.
I20260925 02:43:07.800444 16212 tablet_bootstrap.cc:654] T 400541e581fe412a94781ad06cd21167 P bf02cb03155045689c3075ab6290276d: Neither blocks nor log segments found. Creating new log.
I20260925 02:43:07.801185 16213 log.cc:824] T 400541e581fe412a94781ad06cd21167 P 7e2bc888fce84507ba5be9f8efdbf554: Log is configured to *not* fsync() on all Append() calls
I20260925 02:43:07.801318 16214 tablet_bootstrap.cc:492] T 400541e581fe412a94781ad06cd21167 P aa9d93662a1a4a5dbb0bd66f472479eb: Bootstrap starting.
I20260925 02:43:07.801968 16212 log.cc:824] T 400541e581fe412a94781ad06cd21167 P bf02cb03155045689c3075ab6290276d: Log is configured to *not* fsync() on all Append() calls
I20260925 02:43:07.803254 16213 tablet_bootstrap.cc:492] T 400541e581fe412a94781ad06cd21167 P 7e2bc888fce84507ba5be9f8efdbf554: No bootstrap required, opened a new log
I20260925 02:43:07.803269 16214 tablet_bootstrap.cc:654] T 400541e581fe412a94781ad06cd21167 P aa9d93662a1a4a5dbb0bd66f472479eb: Neither blocks nor log segments found. Creating new log.
I20260925 02:43:07.803447 16213 ts_tablet_manager.cc:1397] T 400541e581fe412a94781ad06cd21167 P 7e2bc888fce84507ba5be9f8efdbf554: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260925 02:43:07.803877 16212 tablet_bootstrap.cc:492] T 400541e581fe412a94781ad06cd21167 P bf02cb03155045689c3075ab6290276d: No bootstrap required, opened a new log
I20260925 02:43:07.803990 16214 log.cc:824] T 400541e581fe412a94781ad06cd21167 P aa9d93662a1a4a5dbb0bd66f472479eb: Log is configured to *not* fsync() on all Append() calls
I20260925 02:43:07.804076 16212 ts_tablet_manager.cc:1397] T 400541e581fe412a94781ad06cd21167 P bf02cb03155045689c3075ab6290276d: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260925 02:43:07.805599 16214 tablet_bootstrap.cc:492] T 400541e581fe412a94781ad06cd21167 P aa9d93662a1a4a5dbb0bd66f472479eb: No bootstrap required, opened a new log
I20260925 02:43:07.805773 16214 ts_tablet_manager.cc:1397] T 400541e581fe412a94781ad06cd21167 P aa9d93662a1a4a5dbb0bd66f472479eb: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260925 02:43:07.807397 16213 raft_consensus.cc:348] T 400541e581fe412a94781ad06cd21167 P 7e2bc888fce84507ba5be9f8efdbf554 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e2bc888fce84507ba5be9f8efdbf554" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 33691 } } peers { permanent_uuid: "aa9d93662a1a4a5dbb0bd66f472479eb" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33685 } } peers { permanent_uuid: "bf02cb03155045689c3075ab6290276d" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 41557 } }
I20260925 02:43:07.807677 16213 raft_consensus.cc:374] T 400541e581fe412a94781ad06cd21167 P 7e2bc888fce84507ba5be9f8efdbf554 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:43:07.807735 16213 raft_consensus.cc:729] T 400541e581fe412a94781ad06cd21167 P 7e2bc888fce84507ba5be9f8efdbf554 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e2bc888fce84507ba5be9f8efdbf554, State: Initialized, Role: FOLLOWER
I20260925 02:43:07.807740 16212 raft_consensus.cc:348] T 400541e581fe412a94781ad06cd21167 P bf02cb03155045689c3075ab6290276d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e2bc888fce84507ba5be9f8efdbf554" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 33691 } } peers { permanent_uuid: "aa9d93662a1a4a5dbb0bd66f472479eb" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33685 } } peers { permanent_uuid: "bf02cb03155045689c3075ab6290276d" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 41557 } }
I20260925 02:43:07.807969 16212 raft_consensus.cc:374] T 400541e581fe412a94781ad06cd21167 P bf02cb03155045689c3075ab6290276d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:43:07.808013 16212 raft_consensus.cc:729] T 400541e581fe412a94781ad06cd21167 P bf02cb03155045689c3075ab6290276d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf02cb03155045689c3075ab6290276d, State: Initialized, Role: FOLLOWER
I20260925 02:43:07.808202 16213 consensus_queue.cc:260] T 400541e581fe412a94781ad06cd21167 P 7e2bc888fce84507ba5be9f8efdbf554 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e2bc888fce84507ba5be9f8efdbf554" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 33691 } } peers { permanent_uuid: "aa9d93662a1a4a5dbb0bd66f472479eb" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33685 } } peers { permanent_uuid: "bf02cb03155045689c3075ab6290276d" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 41557 } }
I20260925 02:43:07.808398 16212 consensus_queue.cc:260] T 400541e581fe412a94781ad06cd21167 P bf02cb03155045689c3075ab6290276d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e2bc888fce84507ba5be9f8efdbf554" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 33691 } } peers { permanent_uuid: "aa9d93662a1a4a5dbb0bd66f472479eb" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33685 } } peers { permanent_uuid: "bf02cb03155045689c3075ab6290276d" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 41557 } }
I20260925 02:43:07.808910 16189 heartbeater.cc:503] Master 127.13.157.190:41363 was elected leader, sending a full tablet report...
I20260925 02:43:07.809245 16212 ts_tablet_manager.cc:1428] T 400541e581fe412a94781ad06cd21167 P bf02cb03155045689c3075ab6290276d: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260925 02:43:07.809245 16213 ts_tablet_manager.cc:1428] T 400541e581fe412a94781ad06cd21167 P 7e2bc888fce84507ba5be9f8efdbf554: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260925 02:43:07.809569 15901 heartbeater.cc:503] Master 127.13.157.190:41363 was elected leader, sending a full tablet report...
I20260925 02:43:07.809686 16214 raft_consensus.cc:348] T 400541e581fe412a94781ad06cd21167 P aa9d93662a1a4a5dbb0bd66f472479eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e2bc888fce84507ba5be9f8efdbf554" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 33691 } } peers { permanent_uuid: "aa9d93662a1a4a5dbb0bd66f472479eb" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33685 } } peers { permanent_uuid: "bf02cb03155045689c3075ab6290276d" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 41557 } }
I20260925 02:43:07.809913 16214 raft_consensus.cc:374] T 400541e581fe412a94781ad06cd21167 P aa9d93662a1a4a5dbb0bd66f472479eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:43:07.809962 16214 raft_consensus.cc:729] T 400541e581fe412a94781ad06cd21167 P aa9d93662a1a4a5dbb0bd66f472479eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa9d93662a1a4a5dbb0bd66f472479eb, State: Initialized, Role: FOLLOWER
I20260925 02:43:07.810361 16214 consensus_queue.cc:260] T 400541e581fe412a94781ad06cd21167 P aa9d93662a1a4a5dbb0bd66f472479eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e2bc888fce84507ba5be9f8efdbf554" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 33691 } } peers { permanent_uuid: "aa9d93662a1a4a5dbb0bd66f472479eb" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33685 } } peers { permanent_uuid: "bf02cb03155045689c3075ab6290276d" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 41557 } }
I20260925 02:43:07.811004 16046 heartbeater.cc:503] Master 127.13.157.190:41363 was elected leader, sending a full tablet report...
I20260925 02:43:07.812218 16214 ts_tablet_manager.cc:1428] T 400541e581fe412a94781ad06cd21167 P aa9d93662a1a4a5dbb0bd66f472479eb: Time spent starting tablet: real 0.006s user 0.001s sys 0.005s
W20260925 02:43:07.980989 16190 tablet.cc:2369] T 400541e581fe412a94781ad06cd21167 P 7e2bc888fce84507ba5be9f8efdbf554: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 02:43:08.017831 16047 tablet.cc:2369] T 400541e581fe412a94781ad06cd21167 P aa9d93662a1a4a5dbb0bd66f472479eb: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 02:43:08.034204 15902 tablet.cc:2369] T 400541e581fe412a94781ad06cd21167 P bf02cb03155045689c3075ab6290276d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 02:43:08.133378 16221 raft_consensus.cc:482] T 400541e581fe412a94781ad06cd21167 P aa9d93662a1a4a5dbb0bd66f472479eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 02:43:08.133612 16221 raft_consensus.cc:504] T 400541e581fe412a94781ad06cd21167 P aa9d93662a1a4a5dbb0bd66f472479eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e2bc888fce84507ba5be9f8efdbf554" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 33691 } } peers { permanent_uuid: "aa9d93662a1a4a5dbb0bd66f472479eb" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33685 } } peers { permanent_uuid: "bf02cb03155045689c3075ab6290276d" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 41557 } }
I20260925 02:43:08.134821 16221 leader_election.cc:288] T 400541e581fe412a94781ad06cd21167 P aa9d93662a1a4a5dbb0bd66f472479eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e2bc888fce84507ba5be9f8efdbf554 (127.13.157.131:33691), bf02cb03155045689c3075ab6290276d (127.13.157.129:41557)
I20260925 02:43:08.139462 16223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:08.135494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.129:41557 (local address 127.13.157.130:44511)
0925 02:43:08.135710 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:08.135721 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:08.135793 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:08.136019 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:08.136021 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:08.136037 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:43:08.136118 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:08.136122 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:08.137023 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:08.137026 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:08.138261 (+ 1235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:08.138272 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:08.139090 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:08.139093 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:08.139169 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:08.139191 (+ 22us) client_negotiation.cc:769] Sending connection context
0925 02:43:08.139269 (+ 78us) client_negotiation.cc:240] Negotiation successful
0925 02:43:08.139324 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"mutex_wait_us":62,"thread_start_us":73,"threads_started":1}
I20260925 02:43:08.139509 16210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:08.135617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.130:44511 (local address 127.13.157.129:41557)
0925 02:43:08.135771 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:08.135776 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:08.135790 (+ 14us) server_negotiation.cc:407] Connection header received
0925 02:43:08.135898 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:08.135900 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:08.135938 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:08.135995 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:43:08.136206 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:08.136942 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:08.138367 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:08.138978 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:08.139096 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:08.139149 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:08.139304 (+ 155us) server_negotiation.cc:299] Negotiation successful
0925 02:43:08.139376 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260925 02:43:08.139868 16222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:08.135129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.131:33691 (local address 127.13.157.130:34995)
0925 02:43:08.135385 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:08.135398 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:08.135520 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:08.135802 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:08.135805 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:08.135819 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:43:08.135940 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:08.135945 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:08.136928 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:08.136930 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:08.138261 (+ 1331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:08.138272 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:08.139456 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:08.139460 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:08.139547 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:08.139569 (+ 22us) client_negotiation.cc:769] Sending connection context
0925 02:43:08.139638 (+ 69us) client_negotiation.cc:240] Negotiation successful
0925 02:43:08.139708 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"mutex_wait_us":6,"thread_start_us":80,"threads_started":1}
I20260925 02:43:08.140096 16211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:08.135223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.130:34995 (local address 127.13.157.131:33691)
0925 02:43:08.135378 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:08.135383 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:08.135476 (+ 93us) server_negotiation.cc:407] Connection header received
0925 02:43:08.135633 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:08.135637 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:08.135690 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:08.135772 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:43:08.136043 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:08.136835 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:08.138655 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:08.139340 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:08.139456 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:08.139501 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:08.139894 (+ 393us) server_negotiation.cc:299] Negotiation successful
0925 02:43:08.139977 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260925 02:43:08.140196 15851 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "400541e581fe412a94781ad06cd21167" candidate_uuid: "aa9d93662a1a4a5dbb0bd66f472479eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf02cb03155045689c3075ab6290276d" is_pre_election: true
I20260925 02:43:08.140552 15851 raft_consensus.cc:2454] T 400541e581fe412a94781ad06cd21167 P bf02cb03155045689c3075ab6290276d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa9d93662a1a4a5dbb0bd66f472479eb in term 0.
I20260925 02:43:08.140609 16142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "400541e581fe412a94781ad06cd21167" candidate_uuid: "aa9d93662a1a4a5dbb0bd66f472479eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e2bc888fce84507ba5be9f8efdbf554" is_pre_election: true
I20260925 02:43:08.140892 16142 raft_consensus.cc:2454] T 400541e581fe412a94781ad06cd21167 P 7e2bc888fce84507ba5be9f8efdbf554 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa9d93662a1a4a5dbb0bd66f472479eb in term 0.
I20260925 02:43:08.141023 15929 leader_election.cc:302] T 400541e581fe412a94781ad06cd21167 P aa9d93662a1a4a5dbb0bd66f472479eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aa9d93662a1a4a5dbb0bd66f472479eb, bf02cb03155045689c3075ab6290276d; no voters:
I20260925 02:43:08.141306 16221 raft_consensus.cc:2781] T 400541e581fe412a94781ad06cd21167 P aa9d93662a1a4a5dbb0bd66f472479eb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 02:43:08.141386 16221 raft_consensus.cc:482] T 400541e581fe412a94781ad06cd21167 P aa9d93662a1a4a5dbb0bd66f472479eb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 02:43:08.141438 16221 raft_consensus.cc:3037] T 400541e581fe412a94781ad06cd21167 P aa9d93662a1a4a5dbb0bd66f472479eb [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:43:08.142447 16221 raft_consensus.cc:504] T 400541e581fe412a94781ad06cd21167 P aa9d93662a1a4a5dbb0bd66f472479eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e2bc888fce84507ba5be9f8efdbf554" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 33691 } } peers { permanent_uuid: "aa9d93662a1a4a5dbb0bd66f472479eb" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33685 } } peers { permanent_uuid: "bf02cb03155045689c3075ab6290276d" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 41557 } }
I20260925 02:43:08.142763 16221 leader_election.cc:288] T 400541e581fe412a94781ad06cd21167 P aa9d93662a1a4a5dbb0bd66f472479eb [CANDIDATE]: Term 1 election: Requested vote from peers 7e2bc888fce84507ba5be9f8efdbf554 (127.13.157.131:33691), bf02cb03155045689c3075ab6290276d (127.13.157.129:41557)
I20260925 02:43:08.143117 16142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "400541e581fe412a94781ad06cd21167" candidate_uuid: "aa9d93662a1a4a5dbb0bd66f472479eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e2bc888fce84507ba5be9f8efdbf554"
I20260925 02:43:08.143172 15851 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "400541e581fe412a94781ad06cd21167" candidate_uuid: "aa9d93662a1a4a5dbb0bd66f472479eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf02cb03155045689c3075ab6290276d"
I20260925 02:43:08.143226 16142 raft_consensus.cc:3037] T 400541e581fe412a94781ad06cd21167 P 7e2bc888fce84507ba5be9f8efdbf554 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:43:08.143298 15851 raft_consensus.cc:3037] T 400541e581fe412a94781ad06cd21167 P bf02cb03155045689c3075ab6290276d [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:43:08.144256 15851 raft_consensus.cc:2454] T 400541e581fe412a94781ad06cd21167 P bf02cb03155045689c3075ab6290276d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa9d93662a1a4a5dbb0bd66f472479eb in term 1.
I20260925 02:43:08.144503 16142 raft_consensus.cc:2454] T 400541e581fe412a94781ad06cd21167 P 7e2bc888fce84507ba5be9f8efdbf554 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa9d93662a1a4a5dbb0bd66f472479eb in term 1.
I20260925 02:43:08.144546 15929 leader_election.cc:302] T 400541e581fe412a94781ad06cd21167 P aa9d93662a1a4a5dbb0bd66f472479eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aa9d93662a1a4a5dbb0bd66f472479eb, bf02cb03155045689c3075ab6290276d; no voters:
I20260925 02:43:08.144698 16221 raft_consensus.cc:2781] T 400541e581fe412a94781ad06cd21167 P aa9d93662a1a4a5dbb0bd66f472479eb [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:43:08.144898 16221 raft_consensus.cc:686] T 400541e581fe412a94781ad06cd21167 P aa9d93662a1a4a5dbb0bd66f472479eb [term 1 LEADER]: Becoming Leader. State: Replica: aa9d93662a1a4a5dbb0bd66f472479eb, State: Running, Role: LEADER
I20260925 02:43:08.145210 16221 consensus_queue.cc:237] T 400541e581fe412a94781ad06cd21167 P aa9d93662a1a4a5dbb0bd66f472479eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e2bc888fce84507ba5be9f8efdbf554" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 33691 } } peers { permanent_uuid: "aa9d93662a1a4a5dbb0bd66f472479eb" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33685 } } peers { permanent_uuid: "bf02cb03155045689c3075ab6290276d" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 41557 } }
I20260925 02:43:08.148254 15691 catalog_manager.cc:5495] T 400541e581fe412a94781ad06cd21167 P aa9d93662a1a4a5dbb0bd66f472479eb reported cstate change: term changed from 0 to 1, leader changed from <none> to aa9d93662a1a4a5dbb0bd66f472479eb (127.13.157.130). New cstate: current_term: 1 leader_uuid: "aa9d93662a1a4a5dbb0bd66f472479eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e2bc888fce84507ba5be9f8efdbf554" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 33691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa9d93662a1a4a5dbb0bd66f472479eb" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bf02cb03155045689c3075ab6290276d" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 41557 } health_report { overall_health: UNKNOWN } } }
Sep 25 02:43:08 dist-test-slave-kznw krb5kdc[15646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304186, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.190@KRBTEST.COM for kudu/127.13.157.190@KRBTEST.COM
I20260925 02:43:08.253201 16231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:08.244689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:41363 (local address 127.0.0.1:41444)
0925 02:43:08.244981 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:08.244994 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:08.245101 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:08.245357 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:08.245359 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:08.245752 (+ 393us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:08.245851 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:08.245857 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:08.246707 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:08.246710 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:08.247391 (+ 681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:08.247403 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:08.247902 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:08.247906 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:08.247989 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:08.248553 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:08.248565 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:08.249980 (+ 1415us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:43:08.251615 (+ 1635us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:08.251625 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:08.251632 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:08.251907 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:08.252148 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:08.252151 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:08.252153 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:08.252235 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:08.252595 (+ 360us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:43:08.252602 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:43:08.252711 (+ 109us) client_negotiation.cc:769] Sending connection context
0925 02:43:08.252806 (+ 95us) client_negotiation.cc:240] Negotiation successful
0925 02:43:08.252963 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":93,"threads_started":1}
I20260925 02:43:08.253199 15741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:08.244797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41444 (local address 127.13.157.190:41363)
0925 02:43:08.244913 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:08.244916 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:08.245143 (+ 227us) server_negotiation.cc:407] Connection header received
0925 02:43:08.245195 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:08.245197 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:08.245241 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:08.245324 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:08.245965 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:08.246533 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:08.247519 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:08.247755 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:08.247840 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:08.250159 (+ 2319us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:08.250162 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:08.250164 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:43:08.250185 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:08.251510 (+ 1325us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:08.251990 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:08.251994 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:08.251996 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:08.252065 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:08.252308 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:08.252311 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:08.252313 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:08.252492 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:08.252575 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:08.252865 (+ 290us) server_negotiation.cc:299] Negotiation successful
0925 02:43:08.253016 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":27,"server-negotiator.queue_time_us":43}
I20260925 02:43:08.257278 15691 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.13.157.190@KRBTEST.COM'} at 127.0.0.1:41444:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260925 02:43:08.257995 15691 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 02:43:08.263368 15977 tablet_service.cc:1467] Processing CreateTablet for tablet bf6b75b8e8804b91942c9d1abcbaca21 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=803ea95e974249f5b1b488d852a1c989]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 02:43:08.263175 15831 tablet_service.cc:1467] Processing CreateTablet for tablet bf6b75b8e8804b91942c9d1abcbaca21 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=803ea95e974249f5b1b488d852a1c989]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 02:43:08.263360 16120 tablet_service.cc:1467] Processing CreateTablet for tablet bf6b75b8e8804b91942c9d1abcbaca21 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=803ea95e974249f5b1b488d852a1c989]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 02:43:08.263958 15831 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf6b75b8e8804b91942c9d1abcbaca21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:43:08.263958 15977 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf6b75b8e8804b91942c9d1abcbaca21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:43:08.263976 16120 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf6b75b8e8804b91942c9d1abcbaca21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:43:08.266629 16213 tablet_bootstrap.cc:492] T bf6b75b8e8804b91942c9d1abcbaca21 P 7e2bc888fce84507ba5be9f8efdbf554: Bootstrap starting.
I20260925 02:43:08.266860 16212 tablet_bootstrap.cc:492] T bf6b75b8e8804b91942c9d1abcbaca21 P bf02cb03155045689c3075ab6290276d: Bootstrap starting.
I20260925 02:43:08.267429 16214 tablet_bootstrap.cc:492] T bf6b75b8e8804b91942c9d1abcbaca21 P aa9d93662a1a4a5dbb0bd66f472479eb: Bootstrap starting.
I20260925 02:43:08.267798 16213 tablet_bootstrap.cc:654] T bf6b75b8e8804b91942c9d1abcbaca21 P 7e2bc888fce84507ba5be9f8efdbf554: Neither blocks nor log segments found. Creating new log.
I20260925 02:43:08.267915 16212 tablet_bootstrap.cc:654] T bf6b75b8e8804b91942c9d1abcbaca21 P bf02cb03155045689c3075ab6290276d: Neither blocks nor log segments found. Creating new log.
I20260925 02:43:08.268548 16214 tablet_bootstrap.cc:654] T bf6b75b8e8804b91942c9d1abcbaca21 P aa9d93662a1a4a5dbb0bd66f472479eb: Neither blocks nor log segments found. Creating new log.
I20260925 02:43:08.269371 16213 tablet_bootstrap.cc:492] T bf6b75b8e8804b91942c9d1abcbaca21 P 7e2bc888fce84507ba5be9f8efdbf554: No bootstrap required, opened a new log
I20260925 02:43:08.269469 16212 tablet_bootstrap.cc:492] T bf6b75b8e8804b91942c9d1abcbaca21 P bf02cb03155045689c3075ab6290276d: No bootstrap required, opened a new log
I20260925 02:43:08.269479 16213 ts_tablet_manager.cc:1397] T bf6b75b8e8804b91942c9d1abcbaca21 P 7e2bc888fce84507ba5be9f8efdbf554: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260925 02:43:08.269559 16212 ts_tablet_manager.cc:1397] T bf6b75b8e8804b91942c9d1abcbaca21 P bf02cb03155045689c3075ab6290276d: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260925 02:43:08.269980 16213 raft_consensus.cc:348] T bf6b75b8e8804b91942c9d1abcbaca21 P 7e2bc888fce84507ba5be9f8efdbf554 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf02cb03155045689c3075ab6290276d" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 41557 } } peers { permanent_uuid: "aa9d93662a1a4a5dbb0bd66f472479eb" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33685 } } peers { permanent_uuid: "7e2bc888fce84507ba5be9f8efdbf554" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 33691 } }
I20260925 02:43:08.270010 16212 raft_consensus.cc:348] T bf6b75b8e8804b91942c9d1abcbaca21 P bf02cb03155045689c3075ab6290276d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf02cb03155045689c3075ab6290276d" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 41557 } } peers { permanent_uuid: "aa9d93662a1a4a5dbb0bd66f472479eb" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33685 } } peers { permanent_uuid: "7e2bc888fce84507ba5be9f8efdbf554" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 33691 } }
I20260925 02:43:08.270094 16213 raft_consensus.cc:374] T bf6b75b8e8804b91942c9d1abcbaca21 P 7e2bc888fce84507ba5be9f8efdbf554 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:43:08.270109 16212 raft_consensus.cc:374] T bf6b75b8e8804b91942c9d1abcbaca21 P bf02cb03155045689c3075ab6290276d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:43:08.270121 16213 raft_consensus.cc:729] T bf6b75b8e8804b91942c9d1abcbaca21 P 7e2bc888fce84507ba5be9f8efdbf554 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e2bc888fce84507ba5be9f8efdbf554, State: Initialized, Role: FOLLOWER
I20260925 02:43:08.270149 16212 raft_consensus.cc:729] T bf6b75b8e8804b91942c9d1abcbaca21 P bf02cb03155045689c3075ab6290276d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf02cb03155045689c3075ab6290276d, State: Initialized, Role: FOLLOWER
I20260925 02:43:08.270241 16213 consensus_queue.cc:260] T bf6b75b8e8804b91942c9d1abcbaca21 P 7e2bc888fce84507ba5be9f8efdbf554 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf02cb03155045689c3075ab6290276d" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 41557 } } peers { permanent_uuid: "aa9d93662a1a4a5dbb0bd66f472479eb" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33685 } } peers { permanent_uuid: "7e2bc888fce84507ba5be9f8efdbf554" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 33691 } }
I20260925 02:43:08.270272 16212 consensus_queue.cc:260] T bf6b75b8e8804b91942c9d1abcbaca21 P bf02cb03155045689c3075ab6290276d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf02cb03155045689c3075ab6290276d" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 41557 } } peers { permanent_uuid: "aa9d93662a1a4a5dbb0bd66f472479eb" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33685 } } peers { permanent_uuid: "7e2bc888fce84507ba5be9f8efdbf554" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 33691 } }
I20260925 02:43:08.270519 16212 ts_tablet_manager.cc:1428] T bf6b75b8e8804b91942c9d1abcbaca21 P bf02cb03155045689c3075ab6290276d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260925 02:43:08.270506 16220 tablet_replica.cc:439] T bf6b75b8e8804b91942c9d1abcbaca21 P bf02cb03155045689c3075ab6290276d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf02cb03155045689c3075ab6290276d" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 41557 } } peers { permanent_uuid: "aa9d93662a1a4a5dbb0bd66f472479eb" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33685 } } peers { permanent_uuid: "7e2bc888fce84507ba5be9f8efdbf554" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 33691 } } }
I20260925 02:43:08.270505 16219 tablet_replica.cc:439] T bf6b75b8e8804b91942c9d1abcbaca21 P 7e2bc888fce84507ba5be9f8efdbf554: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf02cb03155045689c3075ab6290276d" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 41557 } } peers { permanent_uuid: "aa9d93662a1a4a5dbb0bd66f472479eb" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33685 } } peers { permanent_uuid: "7e2bc888fce84507ba5be9f8efdbf554" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 33691 } } }
I20260925 02:43:08.270644 16220 tablet_replica.cc:442] T bf6b75b8e8804b91942c9d1abcbaca21 P bf02cb03155045689c3075ab6290276d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 02:43:08.270666 16219 tablet_replica.cc:442] T bf6b75b8e8804b91942c9d1abcbaca21 P 7e2bc888fce84507ba5be9f8efdbf554: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 02:43:08.270515 16213 ts_tablet_manager.cc:1428] T bf6b75b8e8804b91942c9d1abcbaca21 P 7e2bc888fce84507ba5be9f8efdbf554: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260925 02:43:08.271693 16214 tablet_bootstrap.cc:492] T bf6b75b8e8804b91942c9d1abcbaca21 P aa9d93662a1a4a5dbb0bd66f472479eb: No bootstrap required, opened a new log
I20260925 02:43:08.271839 16214 ts_tablet_manager.cc:1397] T bf6b75b8e8804b91942c9d1abcbaca21 P aa9d93662a1a4a5dbb0bd66f472479eb: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.001s
I20260925 02:43:08.272387 16214 raft_consensus.cc:348] T bf6b75b8e8804b91942c9d1abcbaca21 P aa9d93662a1a4a5dbb0bd66f472479eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf02cb03155045689c3075ab6290276d" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 41557 } } peers { permanent_uuid: "aa9d93662a1a4a5dbb0bd66f472479eb" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33685 } } peers { permanent_uuid: "7e2bc888fce84507ba5be9f8efdbf554" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 33691 } }
I20260925 02:43:08.272555 16214 raft_consensus.cc:374] T bf6b75b8e8804b91942c9d1abcbaca21 P aa9d93662a1a4a5dbb0bd66f472479eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:43:08.272646 16214 raft_consensus.cc:729] T bf6b75b8e8804b91942c9d1abcbaca21 P aa9d93662a1a4a5dbb0bd66f472479eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa9d93662a1a4a5dbb0bd66f472479eb, State: Initialized, Role: FOLLOWER
I20260925 02:43:08.272818 16214 consensus_queue.cc:260] T bf6b75b8e8804b91942c9d1abcbaca21 P aa9d93662a1a4a5dbb0bd66f472479eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf02cb03155045689c3075ab6290276d" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 41557 } } peers { permanent_uuid: "aa9d93662a1a4a5dbb0bd66f472479eb" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33685 } } peers { permanent_uuid: "7e2bc888fce84507ba5be9f8efdbf554" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 33691 } }
I20260925 02:43:08.273094 16214 ts_tablet_manager.cc:1428] T bf6b75b8e8804b91942c9d1abcbaca21 P aa9d93662a1a4a5dbb0bd66f472479eb: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260925 02:43:08.273073 16221 tablet_replica.cc:439] T bf6b75b8e8804b91942c9d1abcbaca21 P aa9d93662a1a4a5dbb0bd66f472479eb: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf02cb03155045689c3075ab6290276d" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 41557 } } peers { permanent_uuid: "aa9d93662a1a4a5dbb0bd66f472479eb" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33685 } } peers { permanent_uuid: "7e2bc888fce84507ba5be9f8efdbf554" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 33691 } } }
I20260925 02:43:08.273192 16221 tablet_replica.cc:442] T bf6b75b8e8804b91942c9d1abcbaca21 P aa9d93662a1a4a5dbb0bd66f472479eb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 02:43:08.321676 16220 raft_consensus.cc:482] T bf6b75b8e8804b91942c9d1abcbaca21 P bf02cb03155045689c3075ab6290276d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 02:43:08.321830 16220 raft_consensus.cc:504] T bf6b75b8e8804b91942c9d1abcbaca21 P bf02cb03155045689c3075ab6290276d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf02cb03155045689c3075ab6290276d" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 41557 } } peers { permanent_uuid: "aa9d93662a1a4a5dbb0bd66f472479eb" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33685 } } peers { permanent_uuid: "7e2bc888fce84507ba5be9f8efdbf554" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 33691 } }
I20260925 02:43:08.323027 16220 leader_election.cc:288] T bf6b75b8e8804b91942c9d1abcbaca21 P bf02cb03155045689c3075ab6290276d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa9d93662a1a4a5dbb0bd66f472479eb (127.13.157.130:33685), 7e2bc888fce84507ba5be9f8efdbf554 (127.13.157.131:33691)
I20260925 02:43:08.326602 16236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:08.323342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.129:49713 (local address 127.13.157.130:33685)
0925 02:43:08.323698 (+ 356us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:08.323716 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:08.323730 (+ 14us) server_negotiation.cc:407] Connection header received
0925 02:43:08.323782 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:08.323784 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:08.323824 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:08.323895 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:43:08.324126 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:08.324645 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:08.325711 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:08.326106 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:08.326211 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:08.326276 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:08.326400 (+ 124us) server_negotiation.cc:299] Negotiation successful
0925 02:43:08.326470 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":286,"thread_start_us":96,"threads_started":1}
I20260925 02:43:08.326633 16235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:08.323217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.130:33685 (local address 127.13.157.129:49713)
0925 02:43:08.323521 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:08.323536 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:08.323653 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:08.323918 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:08.323920 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:08.323937 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:43:08.324027 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:08.324033 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:08.324754 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:08.324765 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:08.325594 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:08.325604 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:08.326212 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:08.326215 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:08.326295 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:08.326317 (+ 22us) client_negotiation.cc:769] Sending connection context
0925 02:43:08.326378 (+ 61us) client_negotiation.cc:240] Negotiation successful
0925 02:43:08.326447 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233,"mutex_wait_us":18,"thread_start_us":71,"threads_started":1}
I20260925 02:43:08.326654 16211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:08.323431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.129:36369 (local address 127.13.157.131:33691)
0925 02:43:08.323542 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:08.323547 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:08.323770 (+ 223us) server_negotiation.cc:407] Connection header received
0925 02:43:08.323890 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:08.323893 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:08.323937 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:08.323997 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:43:08.324226 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:08.324738 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:08.325761 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:08.326160 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:08.326263 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:08.326313 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:08.326462 (+ 149us) server_negotiation.cc:299] Negotiation successful
0925 02:43:08.326528 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260925 02:43:08.327183 15998 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf6b75b8e8804b91942c9d1abcbaca21" candidate_uuid: "bf02cb03155045689c3075ab6290276d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa9d93662a1a4a5dbb0bd66f472479eb" is_pre_election: true
I20260925 02:43:08.327351 16237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:08.323336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.131:33691 (local address 127.13.157.129:36369)
0925 02:43:08.323674 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:08.323686 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:08.323789 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:08.324018 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:08.324021 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:08.324037 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:43:08.324131 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:08.324137 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:08.324866 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:08.324870 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:08.325645 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:08.325654 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:08.326266 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:08.326270 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:08.326354 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:08.326375 (+ 21us) client_negotiation.cc:769] Sending connection context
0925 02:43:08.326439 (+ 64us) client_negotiation.cc:240] Negotiation successful
0925 02:43:08.326503 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"mutex_wait_us":18,"thread_start_us":91,"threads_started":1}
I20260925 02:43:08.327397 15998 raft_consensus.cc:2454] T bf6b75b8e8804b91942c9d1abcbaca21 P aa9d93662a1a4a5dbb0bd66f472479eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf02cb03155045689c3075ab6290276d in term 0.
I20260925 02:43:08.327890 16142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf6b75b8e8804b91942c9d1abcbaca21" candidate_uuid: "bf02cb03155045689c3075ab6290276d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e2bc888fce84507ba5be9f8efdbf554" is_pre_election: true
I20260925 02:43:08.327888 15780 leader_election.cc:302] T bf6b75b8e8804b91942c9d1abcbaca21 P bf02cb03155045689c3075ab6290276d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aa9d93662a1a4a5dbb0bd66f472479eb, bf02cb03155045689c3075ab6290276d; no voters:
I20260925 02:43:08.328038 16142 raft_consensus.cc:2454] T bf6b75b8e8804b91942c9d1abcbaca21 P 7e2bc888fce84507ba5be9f8efdbf554 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf02cb03155045689c3075ab6290276d in term 0.
I20260925 02:43:08.328164 16220 raft_consensus.cc:2781] T bf6b75b8e8804b91942c9d1abcbaca21 P bf02cb03155045689c3075ab6290276d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 02:43:08.328220 16220 raft_consensus.cc:482] T bf6b75b8e8804b91942c9d1abcbaca21 P bf02cb03155045689c3075ab6290276d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 02:43:08.328255 16220 raft_consensus.cc:3037] T bf6b75b8e8804b91942c9d1abcbaca21 P bf02cb03155045689c3075ab6290276d [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:43:08.329300 16220 raft_consensus.cc:504] T bf6b75b8e8804b91942c9d1abcbaca21 P bf02cb03155045689c3075ab6290276d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf02cb03155045689c3075ab6290276d" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 41557 } } peers { permanent_uuid: "aa9d93662a1a4a5dbb0bd66f472479eb" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33685 } } peers { permanent_uuid: "7e2bc888fce84507ba5be9f8efdbf554" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 33691 } }
I20260925 02:43:08.329613 16220 leader_election.cc:288] T bf6b75b8e8804b91942c9d1abcbaca21 P bf02cb03155045689c3075ab6290276d [CANDIDATE]: Term 1 election: Requested vote from peers aa9d93662a1a4a5dbb0bd66f472479eb (127.13.157.130:33685), 7e2bc888fce84507ba5be9f8efdbf554 (127.13.157.131:33691)
I20260925 02:43:08.329918 15998 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf6b75b8e8804b91942c9d1abcbaca21" candidate_uuid: "bf02cb03155045689c3075ab6290276d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa9d93662a1a4a5dbb0bd66f472479eb"
I20260925 02:43:08.329957 16142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf6b75b8e8804b91942c9d1abcbaca21" candidate_uuid: "bf02cb03155045689c3075ab6290276d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e2bc888fce84507ba5be9f8efdbf554"
I20260925 02:43:08.330019 15998 raft_consensus.cc:3037] T bf6b75b8e8804b91942c9d1abcbaca21 P aa9d93662a1a4a5dbb0bd66f472479eb [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:43:08.330063 16142 raft_consensus.cc:3037] T bf6b75b8e8804b91942c9d1abcbaca21 P 7e2bc888fce84507ba5be9f8efdbf554 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:43:08.331024 15998 raft_consensus.cc:2454] T bf6b75b8e8804b91942c9d1abcbaca21 P aa9d93662a1a4a5dbb0bd66f472479eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf02cb03155045689c3075ab6290276d in term 1.
I20260925 02:43:08.331084 16142 raft_consensus.cc:2454] T bf6b75b8e8804b91942c9d1abcbaca21 P 7e2bc888fce84507ba5be9f8efdbf554 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf02cb03155045689c3075ab6290276d in term 1.
I20260925 02:43:08.331362 15782 leader_election.cc:302] T bf6b75b8e8804b91942c9d1abcbaca21 P bf02cb03155045689c3075ab6290276d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e2bc888fce84507ba5be9f8efdbf554, bf02cb03155045689c3075ab6290276d; no voters:
I20260925 02:43:08.331524 16220 raft_consensus.cc:2781] T bf6b75b8e8804b91942c9d1abcbaca21 P bf02cb03155045689c3075ab6290276d [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:43:08.331736 16220 raft_consensus.cc:686] T bf6b75b8e8804b91942c9d1abcbaca21 P bf02cb03155045689c3075ab6290276d [term 1 LEADER]: Becoming Leader. State: Replica: bf02cb03155045689c3075ab6290276d, State: Running, Role: LEADER
I20260925 02:43:08.332095 16220 consensus_queue.cc:237] T bf6b75b8e8804b91942c9d1abcbaca21 P bf02cb03155045689c3075ab6290276d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf02cb03155045689c3075ab6290276d" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 41557 } } peers { permanent_uuid: "aa9d93662a1a4a5dbb0bd66f472479eb" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33685 } } peers { permanent_uuid: "7e2bc888fce84507ba5be9f8efdbf554" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 33691 } }
I20260925 02:43:08.334410 16238 tablet_replica.cc:439] T bf6b75b8e8804b91942c9d1abcbaca21 P bf02cb03155045689c3075ab6290276d: TxnStatusTablet state changed. Reason: New leader bf02cb03155045689c3075ab6290276d. Latest consensus state: current_term: 1 leader_uuid: "bf02cb03155045689c3075ab6290276d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf02cb03155045689c3075ab6290276d" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 41557 } } peers { permanent_uuid: "aa9d93662a1a4a5dbb0bd66f472479eb" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33685 } } peers { permanent_uuid: "7e2bc888fce84507ba5be9f8efdbf554" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 33691 } } }
I20260925 02:43:08.334525 16238 tablet_replica.cc:442] T bf6b75b8e8804b91942c9d1abcbaca21 P bf02cb03155045689c3075ab6290276d: This TxnStatusTablet replica's current role is: LEADER
I20260925 02:43:08.334954 16240 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260925 02:43:08.335208 16240 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260925 02:43:08.335450 15691 catalog_manager.cc:5495] T bf6b75b8e8804b91942c9d1abcbaca21 P bf02cb03155045689c3075ab6290276d reported cstate change: term changed from 0 to 1, leader changed from <none> to bf02cb03155045689c3075ab6290276d (127.13.157.129). New cstate: current_term: 1 leader_uuid: "bf02cb03155045689c3075ab6290276d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf02cb03155045689c3075ab6290276d" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 41557 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aa9d93662a1a4a5dbb0bd66f472479eb" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e2bc888fce84507ba5be9f8efdbf554" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 33691 } health_report { overall_health: UNKNOWN } } }
I20260925 02:43:08.354096 16231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:08.350062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.129:41557 (local address 127.0.0.1:53560)
0925 02:43:08.350284 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:08.350299 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:08.350416 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:08.350742 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:08.350745 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:08.350753 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 02:43:08.350856 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:08.350861 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:08.351701 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:08.351705 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:08.352467 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:08.352477 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:08.352963 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:08.352967 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:08.353064 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:08.353119 (+ 55us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 02:43:08.353789 (+ 670us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 02:43:08.353803 (+ 14us) client_negotiation.cc:769] Sending connection context
0925 02:43:08.353863 (+ 60us) client_negotiation.cc:240] Negotiation successful
0925 02:43:08.353956 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107}
I20260925 02:43:08.354113 16210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:08.350155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53560 (local address 127.13.157.129:41557)
0925 02:43:08.350270 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:08.350275 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:08.350357 (+ 82us) server_negotiation.cc:407] Connection header received
0925 02:43:08.350547 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:08.350550 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:08.350629 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:08.350714 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 02:43:08.350988 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:08.351602 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:08.352571 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:08.352850 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:08.352961 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:08.353256 (+ 295us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 02:43:08.353696 (+ 440us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 02:43:08.353801 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:08.353885 (+ 84us) server_negotiation.cc:299] Negotiation successful
0925 02:43:08.353942 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260925 02:43:08.359257 15998 raft_consensus.cc:1261] T bf6b75b8e8804b91942c9d1abcbaca21 P aa9d93662a1a4a5dbb0bd66f472479eb [term 1 FOLLOWER]: Refusing update from remote peer bf02cb03155045689c3075ab6290276d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:43:08.359299 16142 raft_consensus.cc:1261] T bf6b75b8e8804b91942c9d1abcbaca21 P 7e2bc888fce84507ba5be9f8efdbf554 [term 1 FOLLOWER]: Refusing update from remote peer bf02cb03155045689c3075ab6290276d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:43:08.359952 16238 consensus_queue.cc:1035] T bf6b75b8e8804b91942c9d1abcbaca21 P bf02cb03155045689c3075ab6290276d [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa9d93662a1a4a5dbb0bd66f472479eb" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:43:08.360152 16220 consensus_queue.cc:1035] T bf6b75b8e8804b91942c9d1abcbaca21 P bf02cb03155045689c3075ab6290276d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e2bc888fce84507ba5be9f8efdbf554" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 33691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:43:08.363363 16221 tablet_replica.cc:439] T bf6b75b8e8804b91942c9d1abcbaca21 P aa9d93662a1a4a5dbb0bd66f472479eb: TxnStatusTablet state changed. Reason: New leader bf02cb03155045689c3075ab6290276d. Latest consensus state: current_term: 1 leader_uuid: "bf02cb03155045689c3075ab6290276d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf02cb03155045689c3075ab6290276d" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 41557 } } peers { permanent_uuid: "aa9d93662a1a4a5dbb0bd66f472479eb" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33685 } } peers { permanent_uuid: "7e2bc888fce84507ba5be9f8efdbf554" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 33691 } } }
I20260925 02:43:08.363492 16221 tablet_replica.cc:442] T bf6b75b8e8804b91942c9d1abcbaca21 P aa9d93662a1a4a5dbb0bd66f472479eb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 02:43:08.363910 16219 tablet_replica.cc:439] T bf6b75b8e8804b91942c9d1abcbaca21 P 7e2bc888fce84507ba5be9f8efdbf554: TxnStatusTablet state changed. Reason: New leader bf02cb03155045689c3075ab6290276d. Latest consensus state: current_term: 1 leader_uuid: "bf02cb03155045689c3075ab6290276d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf02cb03155045689c3075ab6290276d" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 41557 } } peers { permanent_uuid: "aa9d93662a1a4a5dbb0bd66f472479eb" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33685 } } peers { permanent_uuid: "7e2bc888fce84507ba5be9f8efdbf554" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 33691 } } }
I20260925 02:43:08.364020 16219 tablet_replica.cc:442] T bf6b75b8e8804b91942c9d1abcbaca21 P 7e2bc888fce84507ba5be9f8efdbf554: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 02:43:08.367147 16220 tablet_replica.cc:439] T bf6b75b8e8804b91942c9d1abcbaca21 P bf02cb03155045689c3075ab6290276d: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bf02cb03155045689c3075ab6290276d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf02cb03155045689c3075ab6290276d" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 41557 } } peers { permanent_uuid: "aa9d93662a1a4a5dbb0bd66f472479eb" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33685 } } peers { permanent_uuid: "7e2bc888fce84507ba5be9f8efdbf554" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 33691 } } }
I20260925 02:43:08.367266 16220 tablet_replica.cc:442] T bf6b75b8e8804b91942c9d1abcbaca21 P bf02cb03155045689c3075ab6290276d: This TxnStatusTablet replica's current role is: LEADER
I20260925 02:43:08.367530 16238 tablet_replica.cc:439] T bf6b75b8e8804b91942c9d1abcbaca21 P bf02cb03155045689c3075ab6290276d: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bf02cb03155045689c3075ab6290276d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf02cb03155045689c3075ab6290276d" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 41557 } } peers { permanent_uuid: "aa9d93662a1a4a5dbb0bd66f472479eb" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33685 } } peers { permanent_uuid: "7e2bc888fce84507ba5be9f8efdbf554" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 33691 } } }
I20260925 02:43:08.367658 16238 tablet_replica.cc:442] T bf6b75b8e8804b91942c9d1abcbaca21 P bf02cb03155045689c3075ab6290276d: This TxnStatusTablet replica's current role is: LEADER
I20260925 02:43:08.368534 16219 tablet_replica.cc:439] T bf6b75b8e8804b91942c9d1abcbaca21 P 7e2bc888fce84507ba5be9f8efdbf554: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bf02cb03155045689c3075ab6290276d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf02cb03155045689c3075ab6290276d" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 41557 } } peers { permanent_uuid: "aa9d93662a1a4a5dbb0bd66f472479eb" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33685 } } peers { permanent_uuid: "7e2bc888fce84507ba5be9f8efdbf554" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 33691 } } }
I20260925 02:43:08.368649 16219 tablet_replica.cc:442] T bf6b75b8e8804b91942c9d1abcbaca21 P 7e2bc888fce84507ba5be9f8efdbf554: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 02:43:08.368882 16221 tablet_replica.cc:439] T bf6b75b8e8804b91942c9d1abcbaca21 P aa9d93662a1a4a5dbb0bd66f472479eb: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bf02cb03155045689c3075ab6290276d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf02cb03155045689c3075ab6290276d" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 41557 } } peers { permanent_uuid: "aa9d93662a1a4a5dbb0bd66f472479eb" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33685 } } peers { permanent_uuid: "7e2bc888fce84507ba5be9f8efdbf554" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 33691 } } }
I20260925 02:43:08.368988 16221 tablet_replica.cc:442] T bf6b75b8e8804b91942c9d1abcbaca21 P aa9d93662a1a4a5dbb0bd66f472479eb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 02:43:08.368968 16244 mvcc.cc:205] Tried to move back new op lower bound from 7333085955512713216 to 7333085955410939904. Current Snapshot: MvccSnapshot[applied={T|T < 7333085955512713216}]
I20260925 02:43:08.384306 16236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:08.380183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57838 (local address 127.13.157.130:33685)
0925 02:43:08.380313 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:08.380317 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:08.380536 (+ 219us) server_negotiation.cc:407] Connection header received
0925 02:43:08.380760 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:08.380762 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:08.380820 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:08.380907 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 02:43:08.381145 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:08.381675 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:08.382872 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:08.383178 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:08.383265 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:08.383536 (+ 271us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 02:43:08.383965 (+ 429us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 02:43:08.384048 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:08.384123 (+ 75us) server_negotiation.cc:299] Negotiation successful
0925 02:43:08.384175 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260925 02:43:08.396001 16255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:08.392346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.129:41557 (local address 127.13.157.130:39711)
0925 02:43:08.392628 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:08.392642 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:08.392731 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:08.392951 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:08.392953 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:08.392959 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 02:43:08.393064 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:08.393068 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:08.394039 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:08.394042 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:08.394722 (+ 680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:08.394732 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:08.395344 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:08.395347 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:08.395418 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:08.395462 (+ 44us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 02:43:08.395753 (+ 291us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 02:43:08.395757 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 02:43:08.395795 (+ 38us) client_negotiation.cc:240] Negotiation successful
0925 02:43:08.395858 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":92,"threads_started":1}
I20260925 02:43:08.396009 16210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:08.392453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.130:39711 (local address 127.13.157.129:41557)
0925 02:43:08.392584 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:08.392589 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:08.392705 (+ 116us) server_negotiation.cc:407] Connection header received
0925 02:43:08.392817 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:08.392820 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:08.392872 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:08.392948 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 02:43:08.393149 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:08.393937 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:08.394832 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:08.395199 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:08.395332 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:08.395602 (+ 270us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 02:43:08.395684 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 02:43:08.395750 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:08.395819 (+ 69us) server_negotiation.cc:299] Negotiation successful
0925 02:43:08.395874 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260925 02:43:08.402824 16142 raft_consensus.cc:1261] T 400541e581fe412a94781ad06cd21167 P 7e2bc888fce84507ba5be9f8efdbf554 [term 1 FOLLOWER]: Refusing update from remote peer aa9d93662a1a4a5dbb0bd66f472479eb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:43:08.403029 15851 raft_consensus.cc:1261] T 400541e581fe412a94781ad06cd21167 P bf02cb03155045689c3075ab6290276d [term 1 FOLLOWER]: Refusing update from remote peer aa9d93662a1a4a5dbb0bd66f472479eb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:43:08.403304 16224 consensus_queue.cc:1035] T 400541e581fe412a94781ad06cd21167 P aa9d93662a1a4a5dbb0bd66f472479eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e2bc888fce84507ba5be9f8efdbf554" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 33691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:43:08.403663 16224 consensus_queue.cc:1035] T 400541e581fe412a94781ad06cd21167 P aa9d93662a1a4a5dbb0bd66f472479eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf02cb03155045689c3075ab6290276d" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 41557 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:43:08.407729 16245 mvcc.cc:205] Tried to move back new op lower bound from 7333085955693850624 to 7333085954645213184. Current Snapshot: MvccSnapshot[applied={T|T < 7333085955693850624}]
I20260925 02:43:08.409548 16243 mvcc.cc:205] Tried to move back new op lower bound from 7333085955693850624 to 7333085954645213184. Current Snapshot: MvccSnapshot[applied={T|T < 7333085955693850624}]
I20260925 02:43:08.429992 16236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:08.426291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.129:45309 (local address 127.13.157.130:33685)
0925 02:43:08.426403 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:08.426408 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:08.426737 (+ 329us) server_negotiation.cc:407] Connection header received
0925 02:43:08.426868 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:08.426870 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:08.426939 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:08.427014 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 02:43:08.427408 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:08.428059 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:08.428927 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:08.429162 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:08.429249 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:08.429518 (+ 269us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 02:43:08.429612 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 02:43:08.429677 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:08.429800 (+ 123us) server_negotiation.cc:299] Negotiation successful
0925 02:43:08.429857 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260925 02:43:08.430408 16262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:08.426171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.130:33685 (local address 127.13.157.129:45309)
0925 02:43:08.426656 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:08.426672 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:08.426775 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:08.427166 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:08.427173 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:08.427184 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 02:43:08.427300 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:08.427307 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:08.428188 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:08.428190 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:08.428814 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:08.428822 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:08.429263 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:08.429267 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:08.429352 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:08.429397 (+ 45us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 02:43:08.429691 (+ 294us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 02:43:08.429698 (+ 7us) client_negotiation.cc:769] Sending connection context
0925 02:43:08.430173 (+ 475us) client_negotiation.cc:240] Negotiation successful
0925 02:43:08.430256 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":114,"threads_started":1}
I20260925 02:43:08.457983 15691 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41438:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 02:43:08.458185 15691 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41438:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 02:43:08.461150 15691 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fade460214df47e393ddd7d3df34fdfe: Sending DeleteTablet for 3 replicas of tablet 400541e581fe412a94781ad06cd21167
I20260925 02:43:08.462064 16120 tablet_service.cc:1514] Processing DeleteTablet for tablet 400541e581fe412a94781ad06cd21167 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:43:08 UTC) from {username='kudu', principal='kudu/127.13.157.190@KRBTEST.COM'} at 127.0.0.1:40784
I20260925 02:43:08.462193 15831 tablet_service.cc:1514] Processing DeleteTablet for tablet 400541e581fe412a94781ad06cd21167 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:43:08 UTC) from {username='kudu', principal='kudu/127.13.157.190@KRBTEST.COM'} at 127.0.0.1:53550
I20260925 02:43:08.462674 16266 tablet_replica.cc:330] T 400541e581fe412a94781ad06cd21167 P 7e2bc888fce84507ba5be9f8efdbf554: stopping tablet replica
I20260925 02:43:08.463056 16266 raft_consensus.cc:2229] T 400541e581fe412a94781ad06cd21167 P 7e2bc888fce84507ba5be9f8efdbf554 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 02:43:08.463317 16266 raft_consensus.cc:2258] T 400541e581fe412a94781ad06cd21167 P 7e2bc888fce84507ba5be9f8efdbf554 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:43:08.463905 15977 tablet_service.cc:1514] Processing DeleteTablet for tablet 400541e581fe412a94781ad06cd21167 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:43:08 UTC) from {username='kudu', principal='kudu/127.13.157.190@KRBTEST.COM'} at 127.0.0.1:57830
I20260925 02:43:08.464437 16267 tablet_replica.cc:330] T 400541e581fe412a94781ad06cd21167 P bf02cb03155045689c3075ab6290276d: stopping tablet replica
I20260925 02:43:08.464445 15691 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41438:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 02:43:08.464915 15691 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 02:43:08.465055 16267 raft_consensus.cc:2229] T 400541e581fe412a94781ad06cd21167 P bf02cb03155045689c3075ab6290276d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 02:43:08.465294 16267 raft_consensus.cc:2258] T 400541e581fe412a94781ad06cd21167 P bf02cb03155045689c3075ab6290276d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:43:08.466759 16268 tablet_replica.cc:330] T 400541e581fe412a94781ad06cd21167 P aa9d93662a1a4a5dbb0bd66f472479eb: stopping tablet replica
I20260925 02:43:08.466805 16267 ts_tablet_manager.cc:1905] T 400541e581fe412a94781ad06cd21167 P bf02cb03155045689c3075ab6290276d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:43:08.467082 16268 raft_consensus.cc:2229] T 400541e581fe412a94781ad06cd21167 P aa9d93662a1a4a5dbb0bd66f472479eb [term 1 LEADER]: Raft consensus shutting down.
I20260925 02:43:08.467468 16268 raft_consensus.cc:2258] T 400541e581fe412a94781ad06cd21167 P aa9d93662a1a4a5dbb0bd66f472479eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:43:08.470152 15831 tablet_service.cc:1467] Processing CreateTablet for tablet bea2d408982041228ce0b6f67b83a5ea (DEFAULT_TABLE table=test-table [id=7897bb25e65a4effa38575d74abf5a3c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:43:08.470458 15831 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bea2d408982041228ce0b6f67b83a5ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:43:08.471562 16268 ts_tablet_manager.cc:1905] T 400541e581fe412a94781ad06cd21167 P aa9d93662a1a4a5dbb0bd66f472479eb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:43:08.473055 16212 tablet_bootstrap.cc:492] T bea2d408982041228ce0b6f67b83a5ea P bf02cb03155045689c3075ab6290276d: Bootstrap starting.
I20260925 02:43:08.473968 16266 ts_tablet_manager.cc:1905] T 400541e581fe412a94781ad06cd21167 P 7e2bc888fce84507ba5be9f8efdbf554: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:43:08.474051 16212 tablet_bootstrap.cc:654] T bea2d408982041228ce0b6f67b83a5ea P bf02cb03155045689c3075ab6290276d: Neither blocks nor log segments found. Creating new log.
I20260925 02:43:08.475915 16120 tablet_service.cc:1467] Processing CreateTablet for tablet bea2d408982041228ce0b6f67b83a5ea (DEFAULT_TABLE table=test-table [id=7897bb25e65a4effa38575d74abf5a3c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:43:08.476212 16120 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bea2d408982041228ce0b6f67b83a5ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:43:08.477464 16267 ts_tablet_manager.cc:1918] T 400541e581fe412a94781ad06cd21167 P bf02cb03155045689c3075ab6290276d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 02:43:08.477620 16267 log.cc:1196] T 400541e581fe412a94781ad06cd21167 P bf02cb03155045689c3075ab6290276d: Deleting WAL directory at /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790304178325258-13942-0/minicluster-data/ts-0/wal/wals/400541e581fe412a94781ad06cd21167
I20260925 02:43:08.478024 16267 ts_tablet_manager.cc:1939] T 400541e581fe412a94781ad06cd21167 P bf02cb03155045689c3075ab6290276d: Deleting consensus metadata
I20260925 02:43:08.478421 16213 tablet_bootstrap.cc:492] T bea2d408982041228ce0b6f67b83a5ea P 7e2bc888fce84507ba5be9f8efdbf554: Bootstrap starting.
I20260925 02:43:08.478422 16268 ts_tablet_manager.cc:1918] T 400541e581fe412a94781ad06cd21167 P aa9d93662a1a4a5dbb0bd66f472479eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 02:43:08.478513 16268 log.cc:1196] T 400541e581fe412a94781ad06cd21167 P aa9d93662a1a4a5dbb0bd66f472479eb: Deleting WAL directory at /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790304178325258-13942-0/minicluster-data/ts-1/wal/wals/400541e581fe412a94781ad06cd21167
I20260925 02:43:08.478830 16268 ts_tablet_manager.cc:1939] T 400541e581fe412a94781ad06cd21167 P aa9d93662a1a4a5dbb0bd66f472479eb: Deleting consensus metadata
I20260925 02:43:08.478837 15977 tablet_service.cc:1467] Processing CreateTablet for tablet bea2d408982041228ce0b6f67b83a5ea (DEFAULT_TABLE table=test-table [id=7897bb25e65a4effa38575d74abf5a3c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:43:08.479117 15977 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bea2d408982041228ce0b6f67b83a5ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:43:08.479439 16213 tablet_bootstrap.cc:654] T bea2d408982041228ce0b6f67b83a5ea P 7e2bc888fce84507ba5be9f8efdbf554: Neither blocks nor log segments found. Creating new log.
I20260925 02:43:08.481292 15677 catalog_manager.cc:4841] TS bf02cb03155045689c3075ab6290276d (127.13.157.129:41557): tablet 400541e581fe412a94781ad06cd21167 (table test-table [id=65191a6cf8eb43d2bcabca6cfdcb003c]) successfully deleted
I20260925 02:43:08.481484 16266 ts_tablet_manager.cc:1918] T 400541e581fe412a94781ad06cd21167 P 7e2bc888fce84507ba5be9f8efdbf554: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 02:43:08.481575 16266 log.cc:1196] T 400541e581fe412a94781ad06cd21167 P 7e2bc888fce84507ba5be9f8efdbf554: Deleting WAL directory at /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790304178325258-13942-0/minicluster-data/ts-2/wal/wals/400541e581fe412a94781ad06cd21167
I20260925 02:43:08.481861 16266 ts_tablet_manager.cc:1939] T 400541e581fe412a94781ad06cd21167 P 7e2bc888fce84507ba5be9f8efdbf554: Deleting consensus metadata
I20260925 02:43:08.482010 16213 tablet_bootstrap.cc:492] T bea2d408982041228ce0b6f67b83a5ea P 7e2bc888fce84507ba5be9f8efdbf554: No bootstrap required, opened a new log
I20260925 02:43:08.482095 16213 ts_tablet_manager.cc:1397] T bea2d408982041228ce0b6f67b83a5ea P 7e2bc888fce84507ba5be9f8efdbf554: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20260925 02:43:08.482378 16214 tablet_bootstrap.cc:492] T bea2d408982041228ce0b6f67b83a5ea P aa9d93662a1a4a5dbb0bd66f472479eb: Bootstrap starting.
I20260925 02:43:08.482538 16213 raft_consensus.cc:348] T bea2d408982041228ce0b6f67b83a5ea P 7e2bc888fce84507ba5be9f8efdbf554 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf02cb03155045689c3075ab6290276d" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 41557 } } peers { permanent_uuid: "aa9d93662a1a4a5dbb0bd66f472479eb" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33685 } } peers { permanent_uuid: "7e2bc888fce84507ba5be9f8efdbf554" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 33691 } }
I20260925 02:43:08.482649 16213 raft_consensus.cc:374] T bea2d408982041228ce0b6f67b83a5ea P 7e2bc888fce84507ba5be9f8efdbf554 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:43:08.482684 16213 raft_consensus.cc:729] T bea2d408982041228ce0b6f67b83a5ea P 7e2bc888fce84507ba5be9f8efdbf554 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e2bc888fce84507ba5be9f8efdbf554, State: Initialized, Role: FOLLOWER
I20260925 02:43:08.482805 16213 consensus_queue.cc:260] T bea2d408982041228ce0b6f67b83a5ea P 7e2bc888fce84507ba5be9f8efdbf554 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf02cb03155045689c3075ab6290276d" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 41557 } } peers { permanent_uuid: "aa9d93662a1a4a5dbb0bd66f472479eb" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33685 } } peers { permanent_uuid: "7e2bc888fce84507ba5be9f8efdbf554" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 33691 } }
I20260925 02:43:08.483078 16213 ts_tablet_manager.cc:1428] T bea2d408982041228ce0b6f67b83a5ea P 7e2bc888fce84507ba5be9f8efdbf554: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260925 02:43:08.483821 16214 tablet_bootstrap.cc:654] T bea2d408982041228ce0b6f67b83a5ea P aa9d93662a1a4a5dbb0bd66f472479eb: Neither blocks nor log segments found. Creating new log.
I20260925 02:43:08.485386 16214 tablet_bootstrap.cc:492] T bea2d408982041228ce0b6f67b83a5ea P aa9d93662a1a4a5dbb0bd66f472479eb: No bootstrap required, opened a new log
I20260925 02:43:08.485464 16214 ts_tablet_manager.cc:1397] T bea2d408982041228ce0b6f67b83a5ea P aa9d93662a1a4a5dbb0bd66f472479eb: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260925 02:43:08.485886 16214 raft_consensus.cc:348] T bea2d408982041228ce0b6f67b83a5ea P aa9d93662a1a4a5dbb0bd66f472479eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf02cb03155045689c3075ab6290276d" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 41557 } } peers { permanent_uuid: "aa9d93662a1a4a5dbb0bd66f472479eb" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33685 } } peers { permanent_uuid: "7e2bc888fce84507ba5be9f8efdbf554" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 33691 } }
I20260925 02:43:08.485980 16214 raft_consensus.cc:374] T bea2d408982041228ce0b6f67b83a5ea P aa9d93662a1a4a5dbb0bd66f472479eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:43:08.486008 16214 raft_consensus.cc:729] T bea2d408982041228ce0b6f67b83a5ea P aa9d93662a1a4a5dbb0bd66f472479eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa9d93662a1a4a5dbb0bd66f472479eb, State: Initialized, Role: FOLLOWER
I20260925 02:43:08.486130 16214 consensus_queue.cc:260] T bea2d408982041228ce0b6f67b83a5ea P aa9d93662a1a4a5dbb0bd66f472479eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf02cb03155045689c3075ab6290276d" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 41557 } } peers { permanent_uuid: "aa9d93662a1a4a5dbb0bd66f472479eb" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33685 } } peers { permanent_uuid: "7e2bc888fce84507ba5be9f8efdbf554" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 33691 } }
I20260925 02:43:08.486382 16214 ts_tablet_manager.cc:1428] T bea2d408982041228ce0b6f67b83a5ea P aa9d93662a1a4a5dbb0bd66f472479eb: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260925 02:43:08.488675 15677 catalog_manager.cc:4841] TS 7e2bc888fce84507ba5be9f8efdbf554 (127.13.157.131:33691): tablet 400541e581fe412a94781ad06cd21167 (table test-table [id=65191a6cf8eb43d2bcabca6cfdcb003c]) successfully deleted
I20260925 02:43:08.489645 15675 catalog_manager.cc:4841] TS aa9d93662a1a4a5dbb0bd66f472479eb (127.13.157.130:33685): tablet 400541e581fe412a94781ad06cd21167 (table test-table [id=65191a6cf8eb43d2bcabca6cfdcb003c]) successfully deleted
I20260925 02:43:08.490504 16212 tablet_bootstrap.cc:492] T bea2d408982041228ce0b6f67b83a5ea P bf02cb03155045689c3075ab6290276d: No bootstrap required, opened a new log
I20260925 02:43:08.490593 16212 ts_tablet_manager.cc:1397] T bea2d408982041228ce0b6f67b83a5ea P bf02cb03155045689c3075ab6290276d: Time spent bootstrapping tablet: real 0.018s user 0.002s sys 0.000s
I20260925 02:43:08.491034 16212 raft_consensus.cc:348] T bea2d408982041228ce0b6f67b83a5ea P bf02cb03155045689c3075ab6290276d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf02cb03155045689c3075ab6290276d" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 41557 } } peers { permanent_uuid: "aa9d93662a1a4a5dbb0bd66f472479eb" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33685 } } peers { permanent_uuid: "7e2bc888fce84507ba5be9f8efdbf554" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 33691 } }
I20260925 02:43:08.491137 16212 raft_consensus.cc:374] T bea2d408982041228ce0b6f67b83a5ea P bf02cb03155045689c3075ab6290276d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:43:08.491166 16212 raft_consensus.cc:729] T bea2d408982041228ce0b6f67b83a5ea P bf02cb03155045689c3075ab6290276d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf02cb03155045689c3075ab6290276d, State: Initialized, Role: FOLLOWER
I20260925 02:43:08.491300 16212 consensus_queue.cc:260] T bea2d408982041228ce0b6f67b83a5ea P bf02cb03155045689c3075ab6290276d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf02cb03155045689c3075ab6290276d" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 41557 } } peers { permanent_uuid: "aa9d93662a1a4a5dbb0bd66f472479eb" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33685 } } peers { permanent_uuid: "7e2bc888fce84507ba5be9f8efdbf554" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 33691 } }
I20260925 02:43:08.491546 16212 ts_tablet_manager.cc:1428] T bea2d408982041228ce0b6f67b83a5ea P bf02cb03155045689c3075ab6290276d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260925 02:43:08.538751 16219 raft_consensus.cc:482] T bea2d408982041228ce0b6f67b83a5ea P 7e2bc888fce84507ba5be9f8efdbf554 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 02:43:08.538879 16219 raft_consensus.cc:504] T bea2d408982041228ce0b6f67b83a5ea P 7e2bc888fce84507ba5be9f8efdbf554 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf02cb03155045689c3075ab6290276d" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 41557 } } peers { permanent_uuid: "aa9d93662a1a4a5dbb0bd66f472479eb" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33685 } } peers { permanent_uuid: "7e2bc888fce84507ba5be9f8efdbf554" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 33691 } }
I20260925 02:43:08.540010 16219 leader_election.cc:288] T bea2d408982041228ce0b6f67b83a5ea P 7e2bc888fce84507ba5be9f8efdbf554 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bf02cb03155045689c3075ab6290276d (127.13.157.129:41557), aa9d93662a1a4a5dbb0bd66f472479eb (127.13.157.130:33685)
I20260925 02:43:08.545007 16272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:08.540247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.129:41557 (local address 127.13.157.131:36355)
0925 02:43:08.540779 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:08.540794 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:08.540906 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:08.541167 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:08.541170 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:08.541185 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:43:08.541339 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:08.541347 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:08.542485 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:08.542488 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:08.543853 (+ 1365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:08.543863 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:08.544589 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:08.544593 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:08.544682 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:08.544705 (+ 23us) client_negotiation.cc:769] Sending connection context
0925 02:43:08.544779 (+ 74us) client_negotiation.cc:240] Negotiation successful
0925 02:43:08.544849 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":233,"threads_started":1}
I20260925 02:43:08.544992 16210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:08.540359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.131:36355 (local address 127.13.157.129:41557)
0925 02:43:08.540491 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:08.540495 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:08.540890 (+ 395us) server_negotiation.cc:407] Connection header received
0925 02:43:08.541018 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:08.541022 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:08.541080 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:08.541149 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:43:08.541501 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:08.542353 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:08.543990 (+ 1637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:08.544463 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:08.544575 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:08.544633 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:08.544809 (+ 176us) server_negotiation.cc:299] Negotiation successful
0925 02:43:08.544886 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260925 02:43:08.545460 16273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:08.540271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.130:33685 (local address 127.13.157.131:45705)
0925 02:43:08.540852 (+ 581us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:08.540866 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:08.541007 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:08.542379 (+ 1372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:08.542382 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:08.542402 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:43:08.542522 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:08.542527 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:08.543267 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:08.543271 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:08.544078 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:08.544088 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:08.545018 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:08.545022 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:08.545124 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:08.545145 (+ 21us) client_negotiation.cc:769] Sending connection context
0925 02:43:08.545240 (+ 95us) client_negotiation.cc:240] Negotiation successful
0925 02:43:08.545308 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":522,"mutex_wait_us":1,"thread_start_us":406,"threads_started":1}
I20260925 02:43:08.545485 16236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:08.540510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.131:45705 (local address 127.13.157.130:33685)
0925 02:43:08.540630 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:08.540635 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:08.540938 (+ 303us) server_negotiation.cc:407] Connection header received
0925 02:43:08.541124 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:08.541128 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:08.541177 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:08.542345 (+ 1168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:43:08.542633 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:08.543166 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:08.544199 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:08.544801 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:08.544912 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:08.544960 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:08.545269 (+ 309us) server_negotiation.cc:299] Negotiation successful
0925 02:43:08.545351 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260925 02:43:08.546095 15998 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bea2d408982041228ce0b6f67b83a5ea" candidate_uuid: "7e2bc888fce84507ba5be9f8efdbf554" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa9d93662a1a4a5dbb0bd66f472479eb" is_pre_election: true
I20260925 02:43:08.546140 15851 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bea2d408982041228ce0b6f67b83a5ea" candidate_uuid: "7e2bc888fce84507ba5be9f8efdbf554" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf02cb03155045689c3075ab6290276d" is_pre_election: true
I20260925 02:43:08.546257 15998 raft_consensus.cc:2454] T bea2d408982041228ce0b6f67b83a5ea P aa9d93662a1a4a5dbb0bd66f472479eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e2bc888fce84507ba5be9f8efdbf554 in term 0.
I20260925 02:43:08.546299 15851 raft_consensus.cc:2454] T bea2d408982041228ce0b6f67b83a5ea P bf02cb03155045689c3075ab6290276d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e2bc888fce84507ba5be9f8efdbf554 in term 0.
I20260925 02:43:08.546667 16071 leader_election.cc:302] T bea2d408982041228ce0b6f67b83a5ea P 7e2bc888fce84507ba5be9f8efdbf554 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e2bc888fce84507ba5be9f8efdbf554, aa9d93662a1a4a5dbb0bd66f472479eb; no voters:
I20260925 02:43:08.546950 16219 raft_consensus.cc:2781] T bea2d408982041228ce0b6f67b83a5ea P 7e2bc888fce84507ba5be9f8efdbf554 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 02:43:08.547019 16219 raft_consensus.cc:482] T bea2d408982041228ce0b6f67b83a5ea P 7e2bc888fce84507ba5be9f8efdbf554 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 02:43:08.547061 16219 raft_consensus.cc:3037] T bea2d408982041228ce0b6f67b83a5ea P 7e2bc888fce84507ba5be9f8efdbf554 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:43:08.548334 16219 raft_consensus.cc:504] T bea2d408982041228ce0b6f67b83a5ea P 7e2bc888fce84507ba5be9f8efdbf554 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf02cb03155045689c3075ab6290276d" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 41557 } } peers { permanent_uuid: "aa9d93662a1a4a5dbb0bd66f472479eb" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33685 } } peers { permanent_uuid: "7e2bc888fce84507ba5be9f8efdbf554" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 33691 } }
I20260925 02:43:08.548722 16219 leader_election.cc:288] T bea2d408982041228ce0b6f67b83a5ea P 7e2bc888fce84507ba5be9f8efdbf554 [CANDIDATE]: Term 1 election: Requested vote from peers bf02cb03155045689c3075ab6290276d (127.13.157.129:41557), aa9d93662a1a4a5dbb0bd66f472479eb (127.13.157.130:33685)
I20260925 02:43:08.549307 15998 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bea2d408982041228ce0b6f67b83a5ea" candidate_uuid: "7e2bc888fce84507ba5be9f8efdbf554" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa9d93662a1a4a5dbb0bd66f472479eb"
I20260925 02:43:08.549319 15851 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bea2d408982041228ce0b6f67b83a5ea" candidate_uuid: "7e2bc888fce84507ba5be9f8efdbf554" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf02cb03155045689c3075ab6290276d"
I20260925 02:43:08.549436 15851 raft_consensus.cc:3037] T bea2d408982041228ce0b6f67b83a5ea P bf02cb03155045689c3075ab6290276d [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:43:08.549436 15998 raft_consensus.cc:3037] T bea2d408982041228ce0b6f67b83a5ea P aa9d93662a1a4a5dbb0bd66f472479eb [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:43:08.550612 15851 raft_consensus.cc:2454] T bea2d408982041228ce0b6f67b83a5ea P bf02cb03155045689c3075ab6290276d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e2bc888fce84507ba5be9f8efdbf554 in term 1.
I20260925 02:43:08.550612 15998 raft_consensus.cc:2454] T bea2d408982041228ce0b6f67b83a5ea P aa9d93662a1a4a5dbb0bd66f472479eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e2bc888fce84507ba5be9f8efdbf554 in term 1.
I20260925 02:43:08.550935 16071 leader_election.cc:302] T bea2d408982041228ce0b6f67b83a5ea P 7e2bc888fce84507ba5be9f8efdbf554 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e2bc888fce84507ba5be9f8efdbf554, aa9d93662a1a4a5dbb0bd66f472479eb; no voters:
I20260925 02:43:08.551245 16219 raft_consensus.cc:2781] T bea2d408982041228ce0b6f67b83a5ea P 7e2bc888fce84507ba5be9f8efdbf554 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:43:08.551450 16219 raft_consensus.cc:686] T bea2d408982041228ce0b6f67b83a5ea P 7e2bc888fce84507ba5be9f8efdbf554 [term 1 LEADER]: Becoming Leader. State: Replica: 7e2bc888fce84507ba5be9f8efdbf554, State: Running, Role: LEADER
I20260925 02:43:08.551715 16219 consensus_queue.cc:237] T bea2d408982041228ce0b6f67b83a5ea P 7e2bc888fce84507ba5be9f8efdbf554 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf02cb03155045689c3075ab6290276d" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 41557 } } peers { permanent_uuid: "aa9d93662a1a4a5dbb0bd66f472479eb" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33685 } } peers { permanent_uuid: "7e2bc888fce84507ba5be9f8efdbf554" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 33691 } }
I20260925 02:43:08.554520 15688 catalog_manager.cc:5495] T bea2d408982041228ce0b6f67b83a5ea P 7e2bc888fce84507ba5be9f8efdbf554 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7e2bc888fce84507ba5be9f8efdbf554 (127.13.157.131). New cstate: current_term: 1 leader_uuid: "7e2bc888fce84507ba5be9f8efdbf554" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf02cb03155045689c3075ab6290276d" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 41557 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa9d93662a1a4a5dbb0bd66f472479eb" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e2bc888fce84507ba5be9f8efdbf554" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 33691 } health_report { overall_health: HEALTHY } } }
I20260925 02:43:08.583657 16211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:08.579720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40790 (local address 127.13.157.131:33691)
0925 02:43:08.579912 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:08.579917 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:08.579952 (+ 35us) server_negotiation.cc:407] Connection header received
0925 02:43:08.580038 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:08.580041 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:08.580108 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:08.580210 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 02:43:08.580434 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:08.581190 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:08.582030 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:08.582357 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:08.582468 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:08.582683 (+ 215us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 02:43:08.583288 (+ 605us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 02:43:08.583395 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:08.583426 (+ 31us) server_negotiation.cc:299] Negotiation successful
0925 02:43:08.583509 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117}
I20260925 02:43:08.595715 16279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:08.591351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.129:41557 (local address 127.13.157.131:48477)
0925 02:43:08.591835 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:08.591848 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:08.591947 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:08.592183 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:08.592184 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:08.592190 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 02:43:08.592763 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:08.592771 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:08.593793 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:08.593795 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:08.594397 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:08.594404 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:08.595026 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:08.595029 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:08.595100 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:08.595148 (+ 48us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 02:43:08.595464 (+ 316us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 02:43:08.595467 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 02:43:08.595509 (+ 42us) client_negotiation.cc:240] Negotiation successful
0925 02:43:08.595572 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":168,"threads_started":1}
I20260925 02:43:08.595724 16210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:08.591512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.131:48477 (local address 127.13.157.129:41557)
0925 02:43:08.591675 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:08.591681 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:08.591924 (+ 243us) server_negotiation.cc:407] Connection header received
0925 02:43:08.592029 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:08.592033 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:08.592092 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:08.592180 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 02:43:08.592918 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:08.593686 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:08.594515 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:08.594853 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:08.595019 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:08.595290 (+ 271us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 02:43:08.595384 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 02:43:08.595466 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:08.595527 (+ 61us) server_negotiation.cc:299] Negotiation successful
0925 02:43:08.595584 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260925 02:43:08.601961 15998 raft_consensus.cc:1261] T bea2d408982041228ce0b6f67b83a5ea P aa9d93662a1a4a5dbb0bd66f472479eb [term 1 FOLLOWER]: Refusing update from remote peer 7e2bc888fce84507ba5be9f8efdbf554: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:43:08.602293 15851 raft_consensus.cc:1261] T bea2d408982041228ce0b6f67b83a5ea P bf02cb03155045689c3075ab6290276d [term 1 FOLLOWER]: Refusing update from remote peer 7e2bc888fce84507ba5be9f8efdbf554: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:43:08.602800 16274 consensus_queue.cc:1035] T bea2d408982041228ce0b6f67b83a5ea P 7e2bc888fce84507ba5be9f8efdbf554 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa9d93662a1a4a5dbb0bd66f472479eb" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:43:08.602983 16219 consensus_queue.cc:1035] T bea2d408982041228ce0b6f67b83a5ea P 7e2bc888fce84507ba5be9f8efdbf554 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf02cb03155045689c3075ab6290276d" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 41557 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:43:08.621385 16262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:08.616605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.131:33691 (local address 127.13.157.129:34821)
0925 02:43:08.616745 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:08.616760 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:08.616833 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:08.617923 (+ 1090us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:08.617926 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:08.617934 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 02:43:08.618041 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:08.618047 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:08.619266 (+ 1219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:08.619269 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:08.620062 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:08.620073 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:08.620619 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:08.620623 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:08.620704 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:08.620743 (+ 39us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 02:43:08.621096 (+ 353us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 02:43:08.621103 (+ 7us) client_negotiation.cc:769] Sending connection context
0925 02:43:08.621144 (+ 41us) client_negotiation.cc:240] Negotiation successful
0925 02:43:08.621246 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260925 02:43:08.621758 16211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:08.616980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.129:34821 (local address 127.13.157.131:33691)
0925 02:43:08.617695 (+ 715us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:08.617700 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:08.617713 (+ 13us) server_negotiation.cc:407] Connection header received
0925 02:43:08.617755 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:08.617757 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:08.617806 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:08.617881 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 02:43:08.618163 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:08.619127 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:08.620196 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:08.620491 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:08.620860 (+ 369us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:08.620922 (+ 62us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 02:43:08.621000 (+ 78us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 02:43:08.621547 (+ 547us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:08.621581 (+ 34us) server_negotiation.cc:299] Negotiation successful
0925 02:43:08.621637 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":640}
W20260925 02:43:08.622529 16245 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260925 02:43:10.183714 15692 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41438:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 02:43:10.183871 15692 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41438:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 02:43:10.186271 15692 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fade460214df47e393ddd7d3df34fdfe: Sending DeleteTablet for 3 replicas of tablet bea2d408982041228ce0b6f67b83a5ea
I20260925 02:43:10.186817 15831 tablet_service.cc:1514] Processing DeleteTablet for tablet bea2d408982041228ce0b6f67b83a5ea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:43:10 UTC) from {username='kudu', principal='kudu/127.13.157.190@KRBTEST.COM'} at 127.0.0.1:53550
I20260925 02:43:10.186882 16120 tablet_service.cc:1514] Processing DeleteTablet for tablet bea2d408982041228ce0b6f67b83a5ea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:43:10 UTC) from {username='kudu', principal='kudu/127.13.157.190@KRBTEST.COM'} at 127.0.0.1:40784
I20260925 02:43:10.186887 15977 tablet_service.cc:1514] Processing DeleteTablet for tablet bea2d408982041228ce0b6f67b83a5ea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:43:10 UTC) from {username='kudu', principal='kudu/127.13.157.190@KRBTEST.COM'} at 127.0.0.1:57830
I20260925 02:43:10.187227 16310 tablet_replica.cc:330] T bea2d408982041228ce0b6f67b83a5ea P bf02cb03155045689c3075ab6290276d: stopping tablet replica
I20260925 02:43:10.187269 16312 tablet_replica.cc:330] T bea2d408982041228ce0b6f67b83a5ea P aa9d93662a1a4a5dbb0bd66f472479eb: stopping tablet replica
I20260925 02:43:10.187362 16312 raft_consensus.cc:2229] T bea2d408982041228ce0b6f67b83a5ea P aa9d93662a1a4a5dbb0bd66f472479eb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 02:43:10.187359 16310 raft_consensus.cc:2229] T bea2d408982041228ce0b6f67b83a5ea P bf02cb03155045689c3075ab6290276d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 02:43:10.187474 16310 raft_consensus.cc:2258] T bea2d408982041228ce0b6f67b83a5ea P bf02cb03155045689c3075ab6290276d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:43:10.187474 16312 raft_consensus.cc:2258] T bea2d408982041228ce0b6f67b83a5ea P aa9d93662a1a4a5dbb0bd66f472479eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:43:10.187690 16311 tablet_replica.cc:330] T bea2d408982041228ce0b6f67b83a5ea P 7e2bc888fce84507ba5be9f8efdbf554: stopping tablet replica
I20260925 02:43:10.187832 16311 raft_consensus.cc:2229] T bea2d408982041228ce0b6f67b83a5ea P 7e2bc888fce84507ba5be9f8efdbf554 [term 1 LEADER]: Raft consensus shutting down.
I20260925 02:43:10.188163 16311 raft_consensus.cc:2258] T bea2d408982041228ce0b6f67b83a5ea P 7e2bc888fce84507ba5be9f8efdbf554 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:43:10.188246 16312 ts_tablet_manager.cc:1905] T bea2d408982041228ce0b6f67b83a5ea P aa9d93662a1a4a5dbb0bd66f472479eb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:43:10.188742 16311 ts_tablet_manager.cc:1905] T bea2d408982041228ce0b6f67b83a5ea P 7e2bc888fce84507ba5be9f8efdbf554: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:43:10.190996 16312 ts_tablet_manager.cc:1918] T bea2d408982041228ce0b6f67b83a5ea P aa9d93662a1a4a5dbb0bd66f472479eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260925 02:43:10.191080 16312 log.cc:1196] T bea2d408982041228ce0b6f67b83a5ea P aa9d93662a1a4a5dbb0bd66f472479eb: Deleting WAL directory at /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790304178325258-13942-0/minicluster-data/ts-1/wal/wals/bea2d408982041228ce0b6f67b83a5ea
I20260925 02:43:10.191341 16312 ts_tablet_manager.cc:1939] T bea2d408982041228ce0b6f67b83a5ea P aa9d93662a1a4a5dbb0bd66f472479eb: Deleting consensus metadata
I20260925 02:43:10.191656 16310 ts_tablet_manager.cc:1905] T bea2d408982041228ce0b6f67b83a5ea P bf02cb03155045689c3075ab6290276d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:43:10.192414 15675 catalog_manager.cc:4841] TS aa9d93662a1a4a5dbb0bd66f472479eb (127.13.157.130:33685): tablet bea2d408982041228ce0b6f67b83a5ea (table test-table [id=7897bb25e65a4effa38575d74abf5a3c]) successfully deleted
I20260925 02:43:10.193702 16310 ts_tablet_manager.cc:1918] T bea2d408982041228ce0b6f67b83a5ea P bf02cb03155045689c3075ab6290276d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260925 02:43:10.193774 16310 log.cc:1196] T bea2d408982041228ce0b6f67b83a5ea P bf02cb03155045689c3075ab6290276d: Deleting WAL directory at /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790304178325258-13942-0/minicluster-data/ts-0/wal/wals/bea2d408982041228ce0b6f67b83a5ea
I20260925 02:43:10.193938 16310 ts_tablet_manager.cc:1939] T bea2d408982041228ce0b6f67b83a5ea P bf02cb03155045689c3075ab6290276d: Deleting consensus metadata
I20260925 02:43:10.194142 16311 ts_tablet_manager.cc:1918] T bea2d408982041228ce0b6f67b83a5ea P 7e2bc888fce84507ba5be9f8efdbf554: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260925 02:43:10.194219 16311 log.cc:1196] T bea2d408982041228ce0b6f67b83a5ea P 7e2bc888fce84507ba5be9f8efdbf554: Deleting WAL directory at /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790304178325258-13942-0/minicluster-data/ts-2/wal/wals/bea2d408982041228ce0b6f67b83a5ea
I20260925 02:43:10.194478 16311 ts_tablet_manager.cc:1939] T bea2d408982041228ce0b6f67b83a5ea P 7e2bc888fce84507ba5be9f8efdbf554: Deleting consensus metadata
I20260925 02:43:10.194697 15677 catalog_manager.cc:4841] TS bf02cb03155045689c3075ab6290276d (127.13.157.129:41557): tablet bea2d408982041228ce0b6f67b83a5ea (table test-table [id=7897bb25e65a4effa38575d74abf5a3c]) successfully deleted
I20260925 02:43:10.195154 15677 catalog_manager.cc:4841] TS 7e2bc888fce84507ba5be9f8efdbf554 (127.13.157.131:33691): tablet bea2d408982041228ce0b6f67b83a5ea (table test-table [id=7897bb25e65a4effa38575d74abf5a3c]) successfully deleted
Sep 25 02:43:10 dist-test-slave-kznw krb5kdc[15646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304190, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 25 02:43:10 dist-test-slave-kznw krb5kdc[15646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304190, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.13.157.190@KRBTEST.COM
I20260925 02:43:10.214309 16320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:10.205539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41446 (local address 127.13.157.190:41363)
0925 02:43:10.205821 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:10.205828 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:10.205900 (+ 72us) server_negotiation.cc:407] Connection header received
0925 02:43:10.206023 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:10.206025 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:10.206071 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:10.206213 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:10.206862 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:10.207498 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:10.208371 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:10.208611 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:10.208696 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:10.211156 (+ 2460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:10.211164 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:10.211166 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:43:10.211191 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:10.212587 (+ 1396us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:10.212986 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:10.212989 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:10.212990 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:10.213122 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:10.213403 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:10.213405 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:10.213406 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:10.213588 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:10.213687 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:10.213944 (+ 257us) server_negotiation.cc:299] Negotiation successful
0925 02:43:10.214124 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":205,"thread_start_us":94,"threads_started":1}
I20260925 02:43:10.216980 15692 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41446:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 02:43:10.217453 15692 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 02:43:10.222427 15977 tablet_service.cc:1467] Processing CreateTablet for tablet 16635a672f024d948ad12eaba984ecca (DEFAULT_TABLE table=test-table [id=ae6cd03d969a480ebe74fd25cf792d33]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:43:10.222561 15831 tablet_service.cc:1467] Processing CreateTablet for tablet 16635a672f024d948ad12eaba984ecca (DEFAULT_TABLE table=test-table [id=ae6cd03d969a480ebe74fd25cf792d33]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:43:10.222548 16120 tablet_service.cc:1467] Processing CreateTablet for tablet 16635a672f024d948ad12eaba984ecca (DEFAULT_TABLE table=test-table [id=ae6cd03d969a480ebe74fd25cf792d33]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:43:10.222796 15977 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 16635a672f024d948ad12eaba984ecca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:43:10.222854 16120 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 16635a672f024d948ad12eaba984ecca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:43:10.222854 15831 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 16635a672f024d948ad12eaba984ecca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:43:10.224993 16321 tablet_bootstrap.cc:492] T 16635a672f024d948ad12eaba984ecca P aa9d93662a1a4a5dbb0bd66f472479eb: Bootstrap starting.
I20260925 02:43:10.225383 16322 tablet_bootstrap.cc:492] T 16635a672f024d948ad12eaba984ecca P 7e2bc888fce84507ba5be9f8efdbf554: Bootstrap starting.
I20260925 02:43:10.226068 16321 tablet_bootstrap.cc:654] T 16635a672f024d948ad12eaba984ecca P aa9d93662a1a4a5dbb0bd66f472479eb: Neither blocks nor log segments found. Creating new log.
I20260925 02:43:10.226212 16322 tablet_bootstrap.cc:654] T 16635a672f024d948ad12eaba984ecca P 7e2bc888fce84507ba5be9f8efdbf554: Neither blocks nor log segments found. Creating new log.
I20260925 02:43:10.226781 16323 tablet_bootstrap.cc:492] T 16635a672f024d948ad12eaba984ecca P bf02cb03155045689c3075ab6290276d: Bootstrap starting.
I20260925 02:43:10.227386 16322 tablet_bootstrap.cc:492] T 16635a672f024d948ad12eaba984ecca P 7e2bc888fce84507ba5be9f8efdbf554: No bootstrap required, opened a new log
I20260925 02:43:10.227483 16322 ts_tablet_manager.cc:1397] T 16635a672f024d948ad12eaba984ecca P 7e2bc888fce84507ba5be9f8efdbf554: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 02:43:10.227644 16323 tablet_bootstrap.cc:654] T 16635a672f024d948ad12eaba984ecca P bf02cb03155045689c3075ab6290276d: Neither blocks nor log segments found. Creating new log.
I20260925 02:43:10.227960 16322 raft_consensus.cc:348] T 16635a672f024d948ad12eaba984ecca P 7e2bc888fce84507ba5be9f8efdbf554 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf02cb03155045689c3075ab6290276d" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 41557 } } peers { permanent_uuid: "aa9d93662a1a4a5dbb0bd66f472479eb" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33685 } } peers { permanent_uuid: "7e2bc888fce84507ba5be9f8efdbf554" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 33691 } }
I20260925 02:43:10.228086 16322 raft_consensus.cc:374] T 16635a672f024d948ad12eaba984ecca P 7e2bc888fce84507ba5be9f8efdbf554 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:43:10.228125 16322 raft_consensus.cc:729] T 16635a672f024d948ad12eaba984ecca P 7e2bc888fce84507ba5be9f8efdbf554 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e2bc888fce84507ba5be9f8efdbf554, State: Initialized, Role: FOLLOWER
I20260925 02:43:10.228283 16322 consensus_queue.cc:260] T 16635a672f024d948ad12eaba984ecca P 7e2bc888fce84507ba5be9f8efdbf554 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf02cb03155045689c3075ab6290276d" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 41557 } } peers { permanent_uuid: "aa9d93662a1a4a5dbb0bd66f472479eb" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33685 } } peers { permanent_uuid: "7e2bc888fce84507ba5be9f8efdbf554" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 33691 } }
I20260925 02:43:10.228565 16322 ts_tablet_manager.cc:1428] T 16635a672f024d948ad12eaba984ecca P 7e2bc888fce84507ba5be9f8efdbf554: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260925 02:43:10.228577 16321 tablet_bootstrap.cc:492] T 16635a672f024d948ad12eaba984ecca P aa9d93662a1a4a5dbb0bd66f472479eb: No bootstrap required, opened a new log
I20260925 02:43:10.228652 16321 ts_tablet_manager.cc:1397] T 16635a672f024d948ad12eaba984ecca P aa9d93662a1a4a5dbb0bd66f472479eb: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260925 02:43:10.228991 16321 raft_consensus.cc:348] T 16635a672f024d948ad12eaba984ecca P aa9d93662a1a4a5dbb0bd66f472479eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf02cb03155045689c3075ab6290276d" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 41557 } } peers { permanent_uuid: "aa9d93662a1a4a5dbb0bd66f472479eb" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33685 } } peers { permanent_uuid: "7e2bc888fce84507ba5be9f8efdbf554" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 33691 } }
I20260925 02:43:10.229076 16321 raft_consensus.cc:374] T 16635a672f024d948ad12eaba984ecca P aa9d93662a1a4a5dbb0bd66f472479eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:43:10.229103 16321 raft_consensus.cc:729] T 16635a672f024d948ad12eaba984ecca P aa9d93662a1a4a5dbb0bd66f472479eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa9d93662a1a4a5dbb0bd66f472479eb, State: Initialized, Role: FOLLOWER
I20260925 02:43:10.229269 16321 consensus_queue.cc:260] T 16635a672f024d948ad12eaba984ecca P aa9d93662a1a4a5dbb0bd66f472479eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf02cb03155045689c3075ab6290276d" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 41557 } } peers { permanent_uuid: "aa9d93662a1a4a5dbb0bd66f472479eb" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33685 } } peers { permanent_uuid: "7e2bc888fce84507ba5be9f8efdbf554" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 33691 } }
I20260925 02:43:10.229650 16321 ts_tablet_manager.cc:1428] T 16635a672f024d948ad12eaba984ecca P aa9d93662a1a4a5dbb0bd66f472479eb: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260925 02:43:10.230137 16323 tablet_bootstrap.cc:492] T 16635a672f024d948ad12eaba984ecca P bf02cb03155045689c3075ab6290276d: No bootstrap required, opened a new log
I20260925 02:43:10.230247 16323 ts_tablet_manager.cc:1397] T 16635a672f024d948ad12eaba984ecca P bf02cb03155045689c3075ab6290276d: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260925 02:43:10.230644 16323 raft_consensus.cc:348] T 16635a672f024d948ad12eaba984ecca P bf02cb03155045689c3075ab6290276d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf02cb03155045689c3075ab6290276d" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 41557 } } peers { permanent_uuid: "aa9d93662a1a4a5dbb0bd66f472479eb" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33685 } } peers { permanent_uuid: "7e2bc888fce84507ba5be9f8efdbf554" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 33691 } }
I20260925 02:43:10.230741 16323 raft_consensus.cc:374] T 16635a672f024d948ad12eaba984ecca P bf02cb03155045689c3075ab6290276d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:43:10.230772 16323 raft_consensus.cc:729] T 16635a672f024d948ad12eaba984ecca P bf02cb03155045689c3075ab6290276d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf02cb03155045689c3075ab6290276d, State: Initialized, Role: FOLLOWER
I20260925 02:43:10.230939 16323 consensus_queue.cc:260] T 16635a672f024d948ad12eaba984ecca P bf02cb03155045689c3075ab6290276d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf02cb03155045689c3075ab6290276d" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 41557 } } peers { permanent_uuid: "aa9d93662a1a4a5dbb0bd66f472479eb" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33685 } } peers { permanent_uuid: "7e2bc888fce84507ba5be9f8efdbf554" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 33691 } }
I20260925 02:43:10.231207 16323 ts_tablet_manager.cc:1428] T 16635a672f024d948ad12eaba984ecca P bf02cb03155045689c3075ab6290276d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260925 02:43:10.537120 16327 raft_consensus.cc:482] T 16635a672f024d948ad12eaba984ecca P aa9d93662a1a4a5dbb0bd66f472479eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 02:43:10.537319 16327 raft_consensus.cc:504] T 16635a672f024d948ad12eaba984ecca P aa9d93662a1a4a5dbb0bd66f472479eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf02cb03155045689c3075ab6290276d" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 41557 } } peers { permanent_uuid: "aa9d93662a1a4a5dbb0bd66f472479eb" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33685 } } peers { permanent_uuid: "7e2bc888fce84507ba5be9f8efdbf554" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 33691 } }
I20260925 02:43:10.537739 16327 leader_election.cc:288] T 16635a672f024d948ad12eaba984ecca P aa9d93662a1a4a5dbb0bd66f472479eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bf02cb03155045689c3075ab6290276d (127.13.157.129:41557), 7e2bc888fce84507ba5be9f8efdbf554 (127.13.157.131:33691)
I20260925 02:43:10.538307 15851 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16635a672f024d948ad12eaba984ecca" candidate_uuid: "aa9d93662a1a4a5dbb0bd66f472479eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf02cb03155045689c3075ab6290276d" is_pre_election: true
I20260925 02:43:10.538447 15851 raft_consensus.cc:2454] T 16635a672f024d948ad12eaba984ecca P bf02cb03155045689c3075ab6290276d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa9d93662a1a4a5dbb0bd66f472479eb in term 0.
I20260925 02:43:10.538478 16141 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16635a672f024d948ad12eaba984ecca" candidate_uuid: "aa9d93662a1a4a5dbb0bd66f472479eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e2bc888fce84507ba5be9f8efdbf554" is_pre_election: true
I20260925 02:43:10.538648 16141 raft_consensus.cc:2454] T 16635a672f024d948ad12eaba984ecca P 7e2bc888fce84507ba5be9f8efdbf554 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa9d93662a1a4a5dbb0bd66f472479eb in term 0.
I20260925 02:43:10.538800 15929 leader_election.cc:302] T 16635a672f024d948ad12eaba984ecca P aa9d93662a1a4a5dbb0bd66f472479eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aa9d93662a1a4a5dbb0bd66f472479eb, bf02cb03155045689c3075ab6290276d; no voters:
I20260925 02:43:10.539011 16327 raft_consensus.cc:2781] T 16635a672f024d948ad12eaba984ecca P aa9d93662a1a4a5dbb0bd66f472479eb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 02:43:10.539073 16327 raft_consensus.cc:482] T 16635a672f024d948ad12eaba984ecca P aa9d93662a1a4a5dbb0bd66f472479eb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 02:43:10.539120 16327 raft_consensus.cc:3037] T 16635a672f024d948ad12eaba984ecca P aa9d93662a1a4a5dbb0bd66f472479eb [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:43:10.540114 16327 raft_consensus.cc:504] T 16635a672f024d948ad12eaba984ecca P aa9d93662a1a4a5dbb0bd66f472479eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf02cb03155045689c3075ab6290276d" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 41557 } } peers { permanent_uuid: "aa9d93662a1a4a5dbb0bd66f472479eb" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33685 } } peers { permanent_uuid: "7e2bc888fce84507ba5be9f8efdbf554" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 33691 } }
I20260925 02:43:10.540424 16327 leader_election.cc:288] T 16635a672f024d948ad12eaba984ecca P aa9d93662a1a4a5dbb0bd66f472479eb [CANDIDATE]: Term 1 election: Requested vote from peers bf02cb03155045689c3075ab6290276d (127.13.157.129:41557), 7e2bc888fce84507ba5be9f8efdbf554 (127.13.157.131:33691)
I20260925 02:43:10.540782 15851 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16635a672f024d948ad12eaba984ecca" candidate_uuid: "aa9d93662a1a4a5dbb0bd66f472479eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf02cb03155045689c3075ab6290276d"
I20260925 02:43:10.540894 15851 raft_consensus.cc:3037] T 16635a672f024d948ad12eaba984ecca P bf02cb03155045689c3075ab6290276d [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:43:10.540838 16141 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16635a672f024d948ad12eaba984ecca" candidate_uuid: "aa9d93662a1a4a5dbb0bd66f472479eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e2bc888fce84507ba5be9f8efdbf554"
I20260925 02:43:10.540975 16141 raft_consensus.cc:3037] T 16635a672f024d948ad12eaba984ecca P 7e2bc888fce84507ba5be9f8efdbf554 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:43:10.541857 15851 raft_consensus.cc:2454] T 16635a672f024d948ad12eaba984ecca P bf02cb03155045689c3075ab6290276d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa9d93662a1a4a5dbb0bd66f472479eb in term 1.
I20260925 02:43:10.542055 16141 raft_consensus.cc:2454] T 16635a672f024d948ad12eaba984ecca P 7e2bc888fce84507ba5be9f8efdbf554 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa9d93662a1a4a5dbb0bd66f472479eb in term 1.
I20260925 02:43:10.542150 15929 leader_election.cc:302] T 16635a672f024d948ad12eaba984ecca P aa9d93662a1a4a5dbb0bd66f472479eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aa9d93662a1a4a5dbb0bd66f472479eb, bf02cb03155045689c3075ab6290276d; no voters:
I20260925 02:43:10.542318 16327 raft_consensus.cc:2781] T 16635a672f024d948ad12eaba984ecca P aa9d93662a1a4a5dbb0bd66f472479eb [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:43:10.542541 16327 raft_consensus.cc:686] T 16635a672f024d948ad12eaba984ecca P aa9d93662a1a4a5dbb0bd66f472479eb [term 1 LEADER]: Becoming Leader. State: Replica: aa9d93662a1a4a5dbb0bd66f472479eb, State: Running, Role: LEADER
I20260925 02:43:10.542697 16327 consensus_queue.cc:237] T 16635a672f024d948ad12eaba984ecca P aa9d93662a1a4a5dbb0bd66f472479eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf02cb03155045689c3075ab6290276d" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 41557 } } peers { permanent_uuid: "aa9d93662a1a4a5dbb0bd66f472479eb" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33685 } } peers { permanent_uuid: "7e2bc888fce84507ba5be9f8efdbf554" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 33691 } }
I20260925 02:43:10.544600 15691 catalog_manager.cc:5495] T 16635a672f024d948ad12eaba984ecca P aa9d93662a1a4a5dbb0bd66f472479eb reported cstate change: term changed from 0 to 1, leader changed from <none> to aa9d93662a1a4a5dbb0bd66f472479eb (127.13.157.130). New cstate: current_term: 1 leader_uuid: "aa9d93662a1a4a5dbb0bd66f472479eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf02cb03155045689c3075ab6290276d" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 41557 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa9d93662a1a4a5dbb0bd66f472479eb" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e2bc888fce84507ba5be9f8efdbf554" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 33691 } health_report { overall_health: UNKNOWN } } }
I20260925 02:43:10.601296 16331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:10.598179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57844 (local address 127.13.157.130:33685)
0925 02:43:10.598421 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:10.598428 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:10.598439 (+ 11us) server_negotiation.cc:407] Connection header received
0925 02:43:10.598476 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:10.598478 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:10.598519 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:10.598608 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 02:43:10.598822 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:10.599400 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:10.600274 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:10.600495 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:10.600596 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:10.600823 (+ 227us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 02:43:10.600906 (+ 83us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 02:43:10.600982 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:10.601119 (+ 137us) server_negotiation.cc:299] Negotiation successful
0925 02:43:10.601168 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":179,"thread_start_us":92,"threads_started":1}
I20260925 02:43:10.607690 16141 raft_consensus.cc:1261] T 16635a672f024d948ad12eaba984ecca P 7e2bc888fce84507ba5be9f8efdbf554 [term 1 FOLLOWER]: Refusing update from remote peer aa9d93662a1a4a5dbb0bd66f472479eb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:43:10.607779 15851 raft_consensus.cc:1261] T 16635a672f024d948ad12eaba984ecca P bf02cb03155045689c3075ab6290276d [term 1 FOLLOWER]: Refusing update from remote peer aa9d93662a1a4a5dbb0bd66f472479eb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:43:10.608105 16327 consensus_queue.cc:1035] T 16635a672f024d948ad12eaba984ecca P aa9d93662a1a4a5dbb0bd66f472479eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf02cb03155045689c3075ab6290276d" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 41557 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:43:10.608270 16328 consensus_queue.cc:1035] T 16635a672f024d948ad12eaba984ecca P aa9d93662a1a4a5dbb0bd66f472479eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e2bc888fce84507ba5be9f8efdbf554" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 33691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:43:10.635984 15691 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41446:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 02:43:10.636135 15691 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41446:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 02:43:10.637959 15691 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fade460214df47e393ddd7d3df34fdfe: Sending DeleteTablet for 3 replicas of tablet 16635a672f024d948ad12eaba984ecca
I20260925 02:43:10.638461 15831 tablet_service.cc:1514] Processing DeleteTablet for tablet 16635a672f024d948ad12eaba984ecca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:43:10 UTC) from {username='kudu', principal='kudu/127.13.157.190@KRBTEST.COM'} at 127.0.0.1:53550
I20260925 02:43:10.638515 15977 tablet_service.cc:1514] Processing DeleteTablet for tablet 16635a672f024d948ad12eaba984ecca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:43:10 UTC) from {username='kudu', principal='kudu/127.13.157.190@KRBTEST.COM'} at 127.0.0.1:57830
I20260925 02:43:10.638512 16120 tablet_service.cc:1514] Processing DeleteTablet for tablet 16635a672f024d948ad12eaba984ecca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:43:10 UTC) from {username='kudu', principal='kudu/127.13.157.190@KRBTEST.COM'} at 127.0.0.1:40784
I20260925 02:43:10.638689 16310 tablet_replica.cc:330] T 16635a672f024d948ad12eaba984ecca P bf02cb03155045689c3075ab6290276d: stopping tablet replica
I20260925 02:43:10.638689 16311 tablet_replica.cc:330] T 16635a672f024d948ad12eaba984ecca P 7e2bc888fce84507ba5be9f8efdbf554: stopping tablet replica
I20260925 02:43:10.638806 16311 raft_consensus.cc:2229] T 16635a672f024d948ad12eaba984ecca P 7e2bc888fce84507ba5be9f8efdbf554 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 02:43:10.638806 16310 raft_consensus.cc:2229] T 16635a672f024d948ad12eaba984ecca P bf02cb03155045689c3075ab6290276d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 02:43:10.638942 16310 raft_consensus.cc:2258] T 16635a672f024d948ad12eaba984ecca P bf02cb03155045689c3075ab6290276d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:43:10.638943 16311 raft_consensus.cc:2258] T 16635a672f024d948ad12eaba984ecca P 7e2bc888fce84507ba5be9f8efdbf554 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:43:10.639006 16312 tablet_replica.cc:330] T 16635a672f024d948ad12eaba984ecca P aa9d93662a1a4a5dbb0bd66f472479eb: stopping tablet replica
I20260925 02:43:10.639137 16312 raft_consensus.cc:2229] T 16635a672f024d948ad12eaba984ecca P aa9d93662a1a4a5dbb0bd66f472479eb [term 1 LEADER]: Raft consensus shutting down.
I20260925 02:43:10.639329 16312 raft_consensus.cc:2258] T 16635a672f024d948ad12eaba984ecca P aa9d93662a1a4a5dbb0bd66f472479eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:43:10.639668 16310 ts_tablet_manager.cc:1905] T 16635a672f024d948ad12eaba984ecca P bf02cb03155045689c3075ab6290276d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:43:10.639668 16311 ts_tablet_manager.cc:1905] T 16635a672f024d948ad12eaba984ecca P 7e2bc888fce84507ba5be9f8efdbf554: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:43:10.640024 16312 ts_tablet_manager.cc:1905] T 16635a672f024d948ad12eaba984ecca P aa9d93662a1a4a5dbb0bd66f472479eb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:43:10.640305 15691 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41446:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 02:43:10.640715 15691 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 02:43:10.642259 16310 ts_tablet_manager.cc:1918] T 16635a672f024d948ad12eaba984ecca P bf02cb03155045689c3075ab6290276d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 02:43:10.642259 16311 ts_tablet_manager.cc:1918] T 16635a672f024d948ad12eaba984ecca P 7e2bc888fce84507ba5be9f8efdbf554: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 02:43:10.642330 16310 log.cc:1196] T 16635a672f024d948ad12eaba984ecca P bf02cb03155045689c3075ab6290276d: Deleting WAL directory at /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790304178325258-13942-0/minicluster-data/ts-0/wal/wals/16635a672f024d948ad12eaba984ecca
I20260925 02:43:10.642330 16311 log.cc:1196] T 16635a672f024d948ad12eaba984ecca P 7e2bc888fce84507ba5be9f8efdbf554: Deleting WAL directory at /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790304178325258-13942-0/minicluster-data/ts-2/wal/wals/16635a672f024d948ad12eaba984ecca
I20260925 02:43:10.642376 16312 ts_tablet_manager.cc:1918] T 16635a672f024d948ad12eaba984ecca P aa9d93662a1a4a5dbb0bd66f472479eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 02:43:10.642431 16312 log.cc:1196] T 16635a672f024d948ad12eaba984ecca P aa9d93662a1a4a5dbb0bd66f472479eb: Deleting WAL directory at /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790304178325258-13942-0/minicluster-data/ts-1/wal/wals/16635a672f024d948ad12eaba984ecca
I20260925 02:43:10.642567 16311 ts_tablet_manager.cc:1939] T 16635a672f024d948ad12eaba984ecca P 7e2bc888fce84507ba5be9f8efdbf554: Deleting consensus metadata
I20260925 02:43:10.642567 16310 ts_tablet_manager.cc:1939] T 16635a672f024d948ad12eaba984ecca P bf02cb03155045689c3075ab6290276d: Deleting consensus metadata
I20260925 02:43:10.642601 16312 ts_tablet_manager.cc:1939] T 16635a672f024d948ad12eaba984ecca P aa9d93662a1a4a5dbb0bd66f472479eb: Deleting consensus metadata
I20260925 02:43:10.643477 15675 catalog_manager.cc:4841] TS aa9d93662a1a4a5dbb0bd66f472479eb (127.13.157.130:33685): tablet 16635a672f024d948ad12eaba984ecca (table test-table [id=ae6cd03d969a480ebe74fd25cf792d33]) successfully deleted
I20260925 02:43:10.643667 15677 catalog_manager.cc:4841] TS 7e2bc888fce84507ba5be9f8efdbf554 (127.13.157.131:33691): tablet 16635a672f024d948ad12eaba984ecca (table test-table [id=ae6cd03d969a480ebe74fd25cf792d33]) successfully deleted
I20260925 02:43:10.643977 15677 catalog_manager.cc:4841] TS bf02cb03155045689c3075ab6290276d (127.13.157.129:41557): tablet 16635a672f024d948ad12eaba984ecca (table test-table [id=ae6cd03d969a480ebe74fd25cf792d33]) successfully deleted
I20260925 02:43:10.646525 15831 tablet_service.cc:1467] Processing CreateTablet for tablet 5070386757e14cf1b7908343f29e9ff2 (DEFAULT_TABLE table=test-table [id=553f27cf055449c49a6eab53e8843f90]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:43:10.646530 16120 tablet_service.cc:1467] Processing CreateTablet for tablet 5070386757e14cf1b7908343f29e9ff2 (DEFAULT_TABLE table=test-table [id=553f27cf055449c49a6eab53e8843f90]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:43:10.646530 15977 tablet_service.cc:1467] Processing CreateTablet for tablet 5070386757e14cf1b7908343f29e9ff2 (DEFAULT_TABLE table=test-table [id=553f27cf055449c49a6eab53e8843f90]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:43:10.646792 15831 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5070386757e14cf1b7908343f29e9ff2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:43:10.646798 15977 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5070386757e14cf1b7908343f29e9ff2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:43:10.646799 16120 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5070386757e14cf1b7908343f29e9ff2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:43:10.648739 16323 tablet_bootstrap.cc:492] T 5070386757e14cf1b7908343f29e9ff2 P bf02cb03155045689c3075ab6290276d: Bootstrap starting.
I20260925 02:43:10.649189 16322 tablet_bootstrap.cc:492] T 5070386757e14cf1b7908343f29e9ff2 P 7e2bc888fce84507ba5be9f8efdbf554: Bootstrap starting.
I20260925 02:43:10.649293 16321 tablet_bootstrap.cc:492] T 5070386757e14cf1b7908343f29e9ff2 P aa9d93662a1a4a5dbb0bd66f472479eb: Bootstrap starting.
I20260925 02:43:10.649633 16323 tablet_bootstrap.cc:654] T 5070386757e14cf1b7908343f29e9ff2 P bf02cb03155045689c3075ab6290276d: Neither blocks nor log segments found. Creating new log.
I20260925 02:43:10.649884 16322 tablet_bootstrap.cc:654] T 5070386757e14cf1b7908343f29e9ff2 P 7e2bc888fce84507ba5be9f8efdbf554: Neither blocks nor log segments found. Creating new log.
I20260925 02:43:10.650107 16321 tablet_bootstrap.cc:654] T 5070386757e14cf1b7908343f29e9ff2 P aa9d93662a1a4a5dbb0bd66f472479eb: Neither blocks nor log segments found. Creating new log.
I20260925 02:43:10.650765 16323 tablet_bootstrap.cc:492] T 5070386757e14cf1b7908343f29e9ff2 P bf02cb03155045689c3075ab6290276d: No bootstrap required, opened a new log
I20260925 02:43:10.650846 16323 ts_tablet_manager.cc:1397] T 5070386757e14cf1b7908343f29e9ff2 P bf02cb03155045689c3075ab6290276d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 02:43:10.651352 16321 tablet_bootstrap.cc:492] T 5070386757e14cf1b7908343f29e9ff2 P aa9d93662a1a4a5dbb0bd66f472479eb: No bootstrap required, opened a new log
I20260925 02:43:10.651288 16323 raft_consensus.cc:348] T 5070386757e14cf1b7908343f29e9ff2 P bf02cb03155045689c3075ab6290276d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf02cb03155045689c3075ab6290276d" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 41557 } } peers { permanent_uuid: "aa9d93662a1a4a5dbb0bd66f472479eb" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33685 } } peers { permanent_uuid: "7e2bc888fce84507ba5be9f8efdbf554" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 33691 } }
I20260925 02:43:10.651403 16323 raft_consensus.cc:374] T 5070386757e14cf1b7908343f29e9ff2 P bf02cb03155045689c3075ab6290276d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:43:10.651432 16323 raft_consensus.cc:729] T 5070386757e14cf1b7908343f29e9ff2 P bf02cb03155045689c3075ab6290276d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf02cb03155045689c3075ab6290276d, State: Initialized, Role: FOLLOWER
I20260925 02:43:10.651444 16321 ts_tablet_manager.cc:1397] T 5070386757e14cf1b7908343f29e9ff2 P aa9d93662a1a4a5dbb0bd66f472479eb: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 02:43:10.651585 16323 consensus_queue.cc:260] T 5070386757e14cf1b7908343f29e9ff2 P bf02cb03155045689c3075ab6290276d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf02cb03155045689c3075ab6290276d" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 41557 } } peers { permanent_uuid: "aa9d93662a1a4a5dbb0bd66f472479eb" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33685 } } peers { permanent_uuid: "7e2bc888fce84507ba5be9f8efdbf554" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 33691 } }
I20260925 02:43:10.651908 16323 ts_tablet_manager.cc:1428] T 5070386757e14cf1b7908343f29e9ff2 P bf02cb03155045689c3075ab6290276d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260925 02:43:10.651837 16321 raft_consensus.cc:348] T 5070386757e14cf1b7908343f29e9ff2 P aa9d93662a1a4a5dbb0bd66f472479eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf02cb03155045689c3075ab6290276d" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 41557 } } peers { permanent_uuid: "aa9d93662a1a4a5dbb0bd66f472479eb" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33685 } } peers { permanent_uuid: "7e2bc888fce84507ba5be9f8efdbf554" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 33691 } }
I20260925 02:43:10.651959 16321 raft_consensus.cc:374] T 5070386757e14cf1b7908343f29e9ff2 P aa9d93662a1a4a5dbb0bd66f472479eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:43:10.651996 16321 raft_consensus.cc:729] T 5070386757e14cf1b7908343f29e9ff2 P aa9d93662a1a4a5dbb0bd66f472479eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa9d93662a1a4a5dbb0bd66f472479eb, State: Initialized, Role: FOLLOWER
I20260925 02:43:10.652125 16321 consensus_queue.cc:260] T 5070386757e14cf1b7908343f29e9ff2 P aa9d93662a1a4a5dbb0bd66f472479eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf02cb03155045689c3075ab6290276d" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 41557 } } peers { permanent_uuid: "aa9d93662a1a4a5dbb0bd66f472479eb" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33685 } } peers { permanent_uuid: "7e2bc888fce84507ba5be9f8efdbf554" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 33691 } }
I20260925 02:43:10.652371 16321 ts_tablet_manager.cc:1428] T 5070386757e14cf1b7908343f29e9ff2 P aa9d93662a1a4a5dbb0bd66f472479eb: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260925 02:43:10.652552 16322 tablet_bootstrap.cc:492] T 5070386757e14cf1b7908343f29e9ff2 P 7e2bc888fce84507ba5be9f8efdbf554: No bootstrap required, opened a new log
I20260925 02:43:10.652663 16322 ts_tablet_manager.cc:1397] T 5070386757e14cf1b7908343f29e9ff2 P 7e2bc888fce84507ba5be9f8efdbf554: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260925 02:43:10.653098 16322 raft_consensus.cc:348] T 5070386757e14cf1b7908343f29e9ff2 P 7e2bc888fce84507ba5be9f8efdbf554 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf02cb03155045689c3075ab6290276d" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 41557 } } peers { permanent_uuid: "aa9d93662a1a4a5dbb0bd66f472479eb" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33685 } } peers { permanent_uuid: "7e2bc888fce84507ba5be9f8efdbf554" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 33691 } }
I20260925 02:43:10.653201 16322 raft_consensus.cc:374] T 5070386757e14cf1b7908343f29e9ff2 P 7e2bc888fce84507ba5be9f8efdbf554 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:43:10.653223 16322 raft_consensus.cc:729] T 5070386757e14cf1b7908343f29e9ff2 P 7e2bc888fce84507ba5be9f8efdbf554 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e2bc888fce84507ba5be9f8efdbf554, State: Initialized, Role: FOLLOWER
I20260925 02:43:10.653337 16322 consensus_queue.cc:260] T 5070386757e14cf1b7908343f29e9ff2 P 7e2bc888fce84507ba5be9f8efdbf554 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf02cb03155045689c3075ab6290276d" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 41557 } } peers { permanent_uuid: "aa9d93662a1a4a5dbb0bd66f472479eb" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33685 } } peers { permanent_uuid: "7e2bc888fce84507ba5be9f8efdbf554" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 33691 } }
I20260925 02:43:10.653570 16322 ts_tablet_manager.cc:1428] T 5070386757e14cf1b7908343f29e9ff2 P 7e2bc888fce84507ba5be9f8efdbf554: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260925 02:43:10.884101 16286 raft_consensus.cc:482] T 5070386757e14cf1b7908343f29e9ff2 P bf02cb03155045689c3075ab6290276d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 02:43:10.884279 16286 raft_consensus.cc:504] T 5070386757e14cf1b7908343f29e9ff2 P bf02cb03155045689c3075ab6290276d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf02cb03155045689c3075ab6290276d" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 41557 } } peers { permanent_uuid: "aa9d93662a1a4a5dbb0bd66f472479eb" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33685 } } peers { permanent_uuid: "7e2bc888fce84507ba5be9f8efdbf554" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 33691 } }
I20260925 02:43:10.884680 16286 leader_election.cc:288] T 5070386757e14cf1b7908343f29e9ff2 P bf02cb03155045689c3075ab6290276d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa9d93662a1a4a5dbb0bd66f472479eb (127.13.157.130:33685), 7e2bc888fce84507ba5be9f8efdbf554 (127.13.157.131:33691)
I20260925 02:43:10.885109 15997 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5070386757e14cf1b7908343f29e9ff2" candidate_uuid: "bf02cb03155045689c3075ab6290276d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa9d93662a1a4a5dbb0bd66f472479eb" is_pre_election: true
I20260925 02:43:10.885094 16141 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5070386757e14cf1b7908343f29e9ff2" candidate_uuid: "bf02cb03155045689c3075ab6290276d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e2bc888fce84507ba5be9f8efdbf554" is_pre_election: true
I20260925 02:43:10.885245 16141 raft_consensus.cc:2454] T 5070386757e14cf1b7908343f29e9ff2 P 7e2bc888fce84507ba5be9f8efdbf554 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf02cb03155045689c3075ab6290276d in term 0.
I20260925 02:43:10.885267 15997 raft_consensus.cc:2454] T 5070386757e14cf1b7908343f29e9ff2 P aa9d93662a1a4a5dbb0bd66f472479eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf02cb03155045689c3075ab6290276d in term 0.
I20260925 02:43:10.885525 15782 leader_election.cc:302] T 5070386757e14cf1b7908343f29e9ff2 P bf02cb03155045689c3075ab6290276d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e2bc888fce84507ba5be9f8efdbf554, bf02cb03155045689c3075ab6290276d; no voters:
I20260925 02:43:10.885704 16286 raft_consensus.cc:2781] T 5070386757e14cf1b7908343f29e9ff2 P bf02cb03155045689c3075ab6290276d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 02:43:10.885771 16286 raft_consensus.cc:482] T 5070386757e14cf1b7908343f29e9ff2 P bf02cb03155045689c3075ab6290276d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 02:43:10.885802 16286 raft_consensus.cc:3037] T 5070386757e14cf1b7908343f29e9ff2 P bf02cb03155045689c3075ab6290276d [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:43:10.886833 16286 raft_consensus.cc:504] T 5070386757e14cf1b7908343f29e9ff2 P bf02cb03155045689c3075ab6290276d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf02cb03155045689c3075ab6290276d" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 41557 } } peers { permanent_uuid: "aa9d93662a1a4a5dbb0bd66f472479eb" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33685 } } peers { permanent_uuid: "7e2bc888fce84507ba5be9f8efdbf554" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 33691 } }
I20260925 02:43:10.887180 16286 leader_election.cc:288] T 5070386757e14cf1b7908343f29e9ff2 P bf02cb03155045689c3075ab6290276d [CANDIDATE]: Term 1 election: Requested vote from peers aa9d93662a1a4a5dbb0bd66f472479eb (127.13.157.130:33685), 7e2bc888fce84507ba5be9f8efdbf554 (127.13.157.131:33691)
I20260925 02:43:10.887557 15997 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5070386757e14cf1b7908343f29e9ff2" candidate_uuid: "bf02cb03155045689c3075ab6290276d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa9d93662a1a4a5dbb0bd66f472479eb"
I20260925 02:43:10.887591 16141 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5070386757e14cf1b7908343f29e9ff2" candidate_uuid: "bf02cb03155045689c3075ab6290276d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e2bc888fce84507ba5be9f8efdbf554"
I20260925 02:43:10.887703 16141 raft_consensus.cc:3037] T 5070386757e14cf1b7908343f29e9ff2 P 7e2bc888fce84507ba5be9f8efdbf554 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:43:10.887703 15997 raft_consensus.cc:3037] T 5070386757e14cf1b7908343f29e9ff2 P aa9d93662a1a4a5dbb0bd66f472479eb [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:43:10.888883 15997 raft_consensus.cc:2454] T 5070386757e14cf1b7908343f29e9ff2 P aa9d93662a1a4a5dbb0bd66f472479eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf02cb03155045689c3075ab6290276d in term 1.
I20260925 02:43:10.888883 16141 raft_consensus.cc:2454] T 5070386757e14cf1b7908343f29e9ff2 P 7e2bc888fce84507ba5be9f8efdbf554 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf02cb03155045689c3075ab6290276d in term 1.
I20260925 02:43:10.889173 15782 leader_election.cc:302] T 5070386757e14cf1b7908343f29e9ff2 P bf02cb03155045689c3075ab6290276d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 7e2bc888fce84507ba5be9f8efdbf554, aa9d93662a1a4a5dbb0bd66f472479eb, bf02cb03155045689c3075ab6290276d; no voters:
I20260925 02:43:10.889340 16286 raft_consensus.cc:2781] T 5070386757e14cf1b7908343f29e9ff2 P bf02cb03155045689c3075ab6290276d [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:43:10.889427 16286 raft_consensus.cc:686] T 5070386757e14cf1b7908343f29e9ff2 P bf02cb03155045689c3075ab6290276d [term 1 LEADER]: Becoming Leader. State: Replica: bf02cb03155045689c3075ab6290276d, State: Running, Role: LEADER
I20260925 02:43:10.889575 16286 consensus_queue.cc:237] T 5070386757e14cf1b7908343f29e9ff2 P bf02cb03155045689c3075ab6290276d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf02cb03155045689c3075ab6290276d" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 41557 } } peers { permanent_uuid: "aa9d93662a1a4a5dbb0bd66f472479eb" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33685 } } peers { permanent_uuid: "7e2bc888fce84507ba5be9f8efdbf554" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 33691 } }
I20260925 02:43:10.891335 15692 catalog_manager.cc:5495] T 5070386757e14cf1b7908343f29e9ff2 P bf02cb03155045689c3075ab6290276d reported cstate change: term changed from 0 to 1, leader changed from <none> to bf02cb03155045689c3075ab6290276d (127.13.157.129). New cstate: current_term: 1 leader_uuid: "bf02cb03155045689c3075ab6290276d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf02cb03155045689c3075ab6290276d" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 41557 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aa9d93662a1a4a5dbb0bd66f472479eb" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e2bc888fce84507ba5be9f8efdbf554" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 33691 } health_report { overall_health: UNKNOWN } } }
I20260925 02:43:10.952759 16340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:10.949575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53566 (local address 127.13.157.129:41557)
0925 02:43:10.949879 (+ 304us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:10.949885 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:10.949895 (+ 10us) server_negotiation.cc:407] Connection header received
0925 02:43:10.949930 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:10.949932 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:10.949972 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:10.950042 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 02:43:10.950303 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:10.950842 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:10.951751 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:10.951983 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:10.952070 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:10.952304 (+ 234us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 02:43:10.952385 (+ 81us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 02:43:10.952442 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:10.952553 (+ 111us) server_negotiation.cc:299] Negotiation successful
0925 02:43:10.952606 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":236,"thread_start_us":123,"threads_started":1}
I20260925 02:43:10.961445 16340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:10.958613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.129:46749 (local address 127.13.157.129:41557)
0925 02:43:10.958729 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:10.958734 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:10.958793 (+ 59us) server_negotiation.cc:407] Connection header received
0925 02:43:10.958941 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:10.958944 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:10.958984 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:10.959039 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 02:43:10.959229 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:10.959696 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:10.960529 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:10.960737 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:10.960817 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:10.961033 (+ 216us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 02:43:10.961094 (+ 61us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 02:43:10.961148 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:10.961249 (+ 101us) server_negotiation.cc:299] Negotiation successful
0925 02:43:10.961303 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":45}
I20260925 02:43:10.961457 16343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:10.958448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.129:41557 (local address 127.13.157.129:46749)
0925 02:43:10.958729 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:10.958743 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:10.958836 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:10.959056 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:10.959059 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:10.959065 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 02:43:10.959145 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:10.959150 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:10.959807 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:10.959810 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:10.960424 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:10.960432 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:10.960829 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:10.960833 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:10.960920 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:10.960940 (+ 20us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 02:43:10.961164 (+ 224us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 02:43:10.961170 (+ 6us) client_negotiation.cc:769] Sending connection context
0925 02:43:10.961240 (+ 70us) client_negotiation.cc:240] Negotiation successful
0925 02:43:10.961337 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"mutex_wait_us":17,"thread_start_us":89,"threads_started":1}
I20260925 02:43:10.967113 15997 raft_consensus.cc:1261] T 5070386757e14cf1b7908343f29e9ff2 P aa9d93662a1a4a5dbb0bd66f472479eb [term 1 FOLLOWER]: Refusing update from remote peer bf02cb03155045689c3075ab6290276d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:43:10.967129 16141 raft_consensus.cc:1261] T 5070386757e14cf1b7908343f29e9ff2 P 7e2bc888fce84507ba5be9f8efdbf554 [term 1 FOLLOWER]: Refusing update from remote peer bf02cb03155045689c3075ab6290276d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:43:10.967471 16290 consensus_queue.cc:1035] T 5070386757e14cf1b7908343f29e9ff2 P bf02cb03155045689c3075ab6290276d [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa9d93662a1a4a5dbb0bd66f472479eb" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:43:10.967599 16286 consensus_queue.cc:1035] T 5070386757e14cf1b7908343f29e9ff2 P bf02cb03155045689c3075ab6290276d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e2bc888fce84507ba5be9f8efdbf554" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 33691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260925 02:43:10.980330 16288 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260925 02:43:12.545503 15692 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41446:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 02:43:12.545708 15692 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41446:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 02:43:12.550644 15692 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fade460214df47e393ddd7d3df34fdfe: Sending DeleteTablet for 3 replicas of tablet 5070386757e14cf1b7908343f29e9ff2
I20260925 02:43:12.551326 15830 tablet_service.cc:1514] Processing DeleteTablet for tablet 5070386757e14cf1b7908343f29e9ff2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:43:12 UTC) from {username='kudu', principal='kudu/127.13.157.190@KRBTEST.COM'} at 127.0.0.1:53550
I20260925 02:43:12.551373 16120 tablet_service.cc:1514] Processing DeleteTablet for tablet 5070386757e14cf1b7908343f29e9ff2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:43:12 UTC) from {username='kudu', principal='kudu/127.13.157.190@KRBTEST.COM'} at 127.0.0.1:40784
I20260925 02:43:12.551416 15977 tablet_service.cc:1514] Processing DeleteTablet for tablet 5070386757e14cf1b7908343f29e9ff2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:43:12 UTC) from {username='kudu', principal='kudu/127.13.157.190@KRBTEST.COM'} at 127.0.0.1:57830
I20260925 02:43:12.551687 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 15764
I20260925 02:43:12.551718 16370 tablet_replica.cc:330] T 5070386757e14cf1b7908343f29e9ff2 P 7e2bc888fce84507ba5be9f8efdbf554: stopping tablet replica
I20260925 02:43:12.553261 16370 raft_consensus.cc:2229] T 5070386757e14cf1b7908343f29e9ff2 P 7e2bc888fce84507ba5be9f8efdbf554 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 02:43:12.553406 16370 raft_consensus.cc:2258] T 5070386757e14cf1b7908343f29e9ff2 P 7e2bc888fce84507ba5be9f8efdbf554 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:43:12.554237 16370 ts_tablet_manager.cc:1905] T 5070386757e14cf1b7908343f29e9ff2 P 7e2bc888fce84507ba5be9f8efdbf554: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:43:12.555848 16372 tablet_replica.cc:330] T 5070386757e14cf1b7908343f29e9ff2 P aa9d93662a1a4a5dbb0bd66f472479eb: stopping tablet replica
I20260925 02:43:12.555994 16372 raft_consensus.cc:2229] T 5070386757e14cf1b7908343f29e9ff2 P aa9d93662a1a4a5dbb0bd66f472479eb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 02:43:12.556142 16372 raft_consensus.cc:2258] T 5070386757e14cf1b7908343f29e9ff2 P aa9d93662a1a4a5dbb0bd66f472479eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:43:12.556864 16372 ts_tablet_manager.cc:1905] T 5070386757e14cf1b7908343f29e9ff2 P aa9d93662a1a4a5dbb0bd66f472479eb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:43:12.557019 16370 ts_tablet_manager.cc:1918] T 5070386757e14cf1b7908343f29e9ff2 P 7e2bc888fce84507ba5be9f8efdbf554: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260925 02:43:12.557093 16370 log.cc:1196] T 5070386757e14cf1b7908343f29e9ff2 P 7e2bc888fce84507ba5be9f8efdbf554: Deleting WAL directory at /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790304178325258-13942-0/minicluster-data/ts-2/wal/wals/5070386757e14cf1b7908343f29e9ff2
I20260925 02:43:12.557346 16370 ts_tablet_manager.cc:1939] T 5070386757e14cf1b7908343f29e9ff2 P 7e2bc888fce84507ba5be9f8efdbf554: Deleting consensus metadata
I20260925 02:43:12.558084 15677 catalog_manager.cc:4841] TS 7e2bc888fce84507ba5be9f8efdbf554 (127.13.157.131:33691): tablet 5070386757e14cf1b7908343f29e9ff2 (table test-table [id=553f27cf055449c49a6eab53e8843f90]) successfully deleted
I20260925 02:43:12.559284 16372 ts_tablet_manager.cc:1918] T 5070386757e14cf1b7908343f29e9ff2 P aa9d93662a1a4a5dbb0bd66f472479eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260925 02:43:12.559347 16372 log.cc:1196] T 5070386757e14cf1b7908343f29e9ff2 P aa9d93662a1a4a5dbb0bd66f472479eb: Deleting WAL directory at /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790304178325258-13942-0/minicluster-data/ts-1/wal/wals/5070386757e14cf1b7908343f29e9ff2
I20260925 02:43:12.559564 16372 ts_tablet_manager.cc:1939] T 5070386757e14cf1b7908343f29e9ff2 P aa9d93662a1a4a5dbb0bd66f472479eb: Deleting consensus metadata
I20260925 02:43:12.560285 15675 catalog_manager.cc:4841] TS aa9d93662a1a4a5dbb0bd66f472479eb (127.13.157.130:33685): tablet 5070386757e14cf1b7908343f29e9ff2 (table test-table [id=553f27cf055449c49a6eab53e8843f90]) successfully deleted
W20260925 02:43:12.562016 15677 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.157.129:41557 (error 108)
I20260925 02:43:12.562830 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 15910
I20260925 02:43:12.563715 16374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:12.562819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.129:41557 (local address 127.0.0.1:45090)
0925 02:43:12.563534 (+ 715us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:12.563611 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.129:41557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":635,"thread_start_us":91,"threads_started":1}
W20260925 02:43:12.563863 15677 catalog_manager.cc:4584] TS bf02cb03155045689c3075ab6290276d (127.13.157.129:41557): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5070386757e14cf1b7908343f29e9ff2: Network error: Client connection negotiation failed: client connection to 127.13.157.129:41557: connect: Connection refused (error 111)
I20260925 02:43:12.570591 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 16055
I20260925 02:43:12.576942 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 15659
2026-09-25T02:43:12Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6889 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790304178325258-13942-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 02:43:12 dist-test-slave-kznw krb5kdc[16377](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 02:43:12 dist-test-slave-kznw krb5kdc[16377](info): set up 2 sockets
Sep 25 02:43:12 dist-test-slave-kznw krb5kdc[16377](info): commencing operation
krb5kdc: starting...
W20260925 02:43:13.538326 13942 mini_kdc.cc:121] Time spent starting KDC: real 0.939s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790304178325258-13942-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790304178325258-13942-0/krb5kdc/test-user.keytab.
Sep 25 02:43:13 dist-test-slave-kznw krb5kdc[16377](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304193, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T02:43:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T02:43:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.157.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.157.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.157.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab.
Entry for principal kudu/127.13.157.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab.
I20260925 02:43:13.658329 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790304178325258-13942-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790304178325258-13942-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790304178325258-13942-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.157.190:46743
--webserver_interface=127.13.157.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab
--principal=kudu/127.13.157.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:44289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.157.190:46743
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:43:13.766340 16390 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:43:13.766623 16390 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:43:13.766665 16390 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:43:13.770900 16390 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:43:13.770992 16390 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:43:13.771020 16390 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:43:13.771078 16390 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:43:13.771135 16390 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:43:13.777060 16390 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:44289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790304178325258-13942-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790304178325258-13942-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.157.190:46743
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab
--principal=kudu/127.13.157.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.157.190:46743
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.157.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16390
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790304178325258-13942-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:43:13.778146 16390 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:43:13.779254 16390 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:43:13.787261 16397 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:13.787281 16399 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:13.787305 16396 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:43:13.787302 16390 server_base.cc:1034] running on GCE node
I20260925 02:43:13.788043 16390 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:43:13.789099 16390 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:43:13.790274 16390 hybrid_clock.cc:648] HybridClock initialized: now 1790304193790248 us; error 37 us; skew 500 ppm
Sep 25 02:43:13 dist-test-slave-kznw krb5kdc[16377](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304193, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:43:13.796537 16390 init.cc:377] Logged in from keytab as kudu/127.13.157.190@KRBTEST.COM (short username kudu)
I20260925 02:43:13.797437 16390 webserver.cc:460] Webserver started at http://127.13.157.190:40413/ using document root <none> and password file <none>
I20260925 02:43:13.797958 16390 fs_manager.cc:359] Metadata directory not provided
I20260925 02:43:13.798014 16390 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:43:13.798182 16390 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:43:13.800232 16390 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790304178325258-13942-0/minicluster-data/master-0/data/instance:
uuid: "f4ab4708c84b4b30a9b8cc8c4d13b743"
format_stamp: "Formatted at 2026-09-25 02:43:13 on dist-test-slave-kznw"
server_key: "caf778c79b7d8c94ab37921e10b95cc8"
server_key_iv: "a6fcf1ef0b5853982a47b8a6de8dd246"
server_key_version: "encryptionkey@0"
I20260925 02:43:13.800750 16390 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790304178325258-13942-0/minicluster-data/master-0/wal/instance:
uuid: "f4ab4708c84b4b30a9b8cc8c4d13b743"
format_stamp: "Formatted at 2026-09-25 02:43:13 on dist-test-slave-kznw"
server_key: "caf778c79b7d8c94ab37921e10b95cc8"
server_key_iv: "a6fcf1ef0b5853982a47b8a6de8dd246"
server_key_version: "encryptionkey@0"
I20260925 02:43:13.804366 16390 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260925 02:43:13.806666 16406 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:13.807870 16390 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260925 02:43:13.807982 16390 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790304178325258-13942-0/minicluster-data/master-0/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790304178325258-13942-0/minicluster-data/master-0/wal
uuid: "f4ab4708c84b4b30a9b8cc8c4d13b743"
format_stamp: "Formatted at 2026-09-25 02:43:13 on dist-test-slave-kznw"
server_key: "caf778c79b7d8c94ab37921e10b95cc8"
server_key_iv: "a6fcf1ef0b5853982a47b8a6de8dd246"
server_key_version: "encryptionkey@0"
I20260925 02:43:13.808071 16390 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790304178325258-13942-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790304178325258-13942-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790304178325258-13942-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:43:13.822211 16390 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:43:13.825084 16390 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:43:13.825256 16390 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:43:13.831878 16390 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.190:46743
I20260925 02:43:13.831920 16460 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.190:46743 every 8 connection(s)
I20260925 02:43:13.832818 16390 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
I20260925 02:43:13.834293 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 16390
I20260925 02:43:13.834390 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790304178325258-13942-0/minicluster-data/master-0/wal/instance
I20260925 02:43:13.834640 13942 external_mini_cluster.cc:1428] Setting key e0dd52edb157a6be811db8343a9376e2
I20260925 02:43:13.835925 16461 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:43:13.842272 16461 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f4ab4708c84b4b30a9b8cc8c4d13b743: Bootstrap starting.
Sep 25 02:43:13 dist-test-slave-kznw krb5kdc[16377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304193, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.157.190@KRBTEST.COM
I20260925 02:43:13.844894 16461 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f4ab4708c84b4b30a9b8cc8c4d13b743: Neither blocks nor log segments found. Creating new log.
I20260925 02:43:13.845598 16461 log.cc:824] T 00000000000000000000000000000000 P f4ab4708c84b4b30a9b8cc8c4d13b743: Log is configured to *not* fsync() on all Append() calls
I20260925 02:43:13.847905 16461 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f4ab4708c84b4b30a9b8cc8c4d13b743: No bootstrap required, opened a new log
I20260925 02:43:13.848778 16464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:13.835967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53386 (local address 127.13.157.190:46743)
0925 02:43:13.836376 (+ 409us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:13.836392 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:13.836428 (+ 36us) server_negotiation.cc:407] Connection header received
0925 02:43:13.837070 (+ 642us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:13.837086 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:13.837410 (+ 324us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:13.837722 (+ 312us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:13.838618 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:13.839728 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:13.840893 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:13.841369 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:13.841498 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:13.844218 (+ 2720us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:13.844223 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:13.844236 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:43:13.844266 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:13.845855 (+ 1589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:13.846268 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:13.846274 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:13.846280 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:13.846406 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:13.846697 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:13.846699 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:13.846700 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:13.847160 (+ 460us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:13.847337 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:13.847730 (+ 393us) server_negotiation.cc:299] Negotiation successful
0925 02:43:13.847989 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":264,"thread_start_us":113,"threads_started":1}
I20260925 02:43:13.850821 16461 raft_consensus.cc:348] T 00000000000000000000000000000000 P f4ab4708c84b4b30a9b8cc8c4d13b743 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4ab4708c84b4b30a9b8cc8c4d13b743" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 46743 } }
I20260925 02:43:13.851070 16461 raft_consensus.cc:374] T 00000000000000000000000000000000 P f4ab4708c84b4b30a9b8cc8c4d13b743 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:43:13.851146 16461 raft_consensus.cc:729] T 00000000000000000000000000000000 P f4ab4708c84b4b30a9b8cc8c4d13b743 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4ab4708c84b4b30a9b8cc8c4d13b743, State: Initialized, Role: FOLLOWER
I20260925 02:43:13.851570 16461 consensus_queue.cc:260] T 00000000000000000000000000000000 P f4ab4708c84b4b30a9b8cc8c4d13b743 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4ab4708c84b4b30a9b8cc8c4d13b743" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 46743 } }
I20260925 02:43:13.851697 16461 raft_consensus.cc:388] T 00000000000000000000000000000000 P f4ab4708c84b4b30a9b8cc8c4d13b743 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 02:43:13.851742 16461 raft_consensus.cc:482] T 00000000000000000000000000000000 P f4ab4708c84b4b30a9b8cc8c4d13b743 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 02:43:13.851810 16461 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f4ab4708c84b4b30a9b8cc8c4d13b743 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:43:13.852628 16461 raft_consensus.cc:504] T 00000000000000000000000000000000 P f4ab4708c84b4b30a9b8cc8c4d13b743 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4ab4708c84b4b30a9b8cc8c4d13b743" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 46743 } }
I20260925 02:43:13.852923 16461 leader_election.cc:302] T 00000000000000000000000000000000 P f4ab4708c84b4b30a9b8cc8c4d13b743 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f4ab4708c84b4b30a9b8cc8c4d13b743; no voters:
I20260925 02:43:13.853189 16461 leader_election.cc:288] T 00000000000000000000000000000000 P f4ab4708c84b4b30a9b8cc8c4d13b743 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 02:43:13.853325 16466 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f4ab4708c84b4b30a9b8cc8c4d13b743 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:43:13.853538 16466 raft_consensus.cc:686] T 00000000000000000000000000000000 P f4ab4708c84b4b30a9b8cc8c4d13b743 [term 1 LEADER]: Becoming Leader. State: Replica: f4ab4708c84b4b30a9b8cc8c4d13b743, State: Running, Role: LEADER
I20260925 02:43:13.853864 16466 consensus_queue.cc:237] T 00000000000000000000000000000000 P f4ab4708c84b4b30a9b8cc8c4d13b743 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4ab4708c84b4b30a9b8cc8c4d13b743" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 46743 } }
I20260925 02:43:13.854312 16461 sys_catalog.cc:564] T 00000000000000000000000000000000 P f4ab4708c84b4b30a9b8cc8c4d13b743 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:43:13.855479 16468 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4ab4708c84b4b30a9b8cc8c4d13b743 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f4ab4708c84b4b30a9b8cc8c4d13b743. Latest consensus state: current_term: 1 leader_uuid: "f4ab4708c84b4b30a9b8cc8c4d13b743" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4ab4708c84b4b30a9b8cc8c4d13b743" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 46743 } } }
I20260925 02:43:13.855633 16468 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4ab4708c84b4b30a9b8cc8c4d13b743 [sys.catalog]: This master's current role is: LEADER
I20260925 02:43:13.855476 16467 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4ab4708c84b4b30a9b8cc8c4d13b743 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f4ab4708c84b4b30a9b8cc8c4d13b743" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4ab4708c84b4b30a9b8cc8c4d13b743" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 46743 } } }
I20260925 02:43:13.855947 16467 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4ab4708c84b4b30a9b8cc8c4d13b743 [sys.catalog]: This master's current role is: LEADER
W20260925 02:43:13.859001 16481 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f4ab4708c84b4b30a9b8cc8c4d13b743: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 02:43:13.859071 16481 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 02:43:13.859177 16475 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:43:13.859866 16475 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:43:13.865396 16475 catalog_manager.cc:1347] Generated new cluster ID: 19960eeb9071463e9d5ef4e9b8b3899f
I20260925 02:43:13.865459 16475 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 02:43:13.879856 16475 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 02:43:13.881187 16475 catalog_manager.cc:1504] Loading token signing keys...
I20260925 02:43:13.892032 16475 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f4ab4708c84b4b30a9b8cc8c4d13b743: Generated new TSK 0
I20260925 02:43:13.892932 16475 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.157.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.157.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.157.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.129.keytab.
Entry for principal kudu/127.13.157.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.129.keytab.
I20260925 02:43:13.928122 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790304178325258-13942-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790304178325258-13942-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790304178325258-13942-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790304178325258-13942-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.157.129:0
--local_ip_for_outbound_sockets=127.13.157.129
--webserver_interface=127.13.157.129
--webserver_port=0
--tserver_master_addrs=127.13.157.190:46743
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.129.keytab
--principal=kudu/127.13.157.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:44289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:43:14.039203 16487 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:43:14.039477 16487 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:43:14.039536 16487 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:43:14.043555 16487 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:43:14.043617 16487 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:43:14.043700 16487 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.157.129
I20260925 02:43:14.048944 16487 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:44289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790304178325258-13942-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790304178325258-13942-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.129.keytab
--principal=kudu/127.13.157.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.157.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790304178325258-13942-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.157.129
--webserver_port=0
--tserver_master_addrs=127.13.157.190:46743
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16487
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.157.129
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790304178325258-13942-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:43:14.050100 16487 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:43:14.051154 16487 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:43:14.060604 16495 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:14.060642 16493 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:14.060647 16492 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:43:14.060963 16487 server_base.cc:1034] running on GCE node
I20260925 02:43:14.061350 16487 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:43:14.061973 16487 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:43:14.063136 16487 hybrid_clock.cc:648] HybridClock initialized: now 1790304194063117 us; error 30 us; skew 500 ppm
Sep 25 02:43:14 dist-test-slave-kznw krb5kdc[16377](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304194, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:43:14.065871 16487 init.cc:377] Logged in from keytab as kudu/127.13.157.129@KRBTEST.COM (short username kudu)
I20260925 02:43:14.066709 16487 webserver.cc:460] Webserver started at http://127.13.157.129:43775/ using document root <none> and password file <none>
I20260925 02:43:14.067322 16487 fs_manager.cc:359] Metadata directory not provided
I20260925 02:43:14.067391 16487 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:43:14.067605 16487 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:43:14.069640 16487 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790304178325258-13942-0/minicluster-data/ts-0/data/instance:
uuid: "869f8d0294dd4190af070a5f009052c4"
format_stamp: "Formatted at 2026-09-25 02:43:14 on dist-test-slave-kznw"
server_key: "40f5a4aa101c7b424cb19bfb598df7d4"
server_key_iv: "1953d39e00a5c604c51156d6f58e5063"
server_key_version: "encryptionkey@0"
I20260925 02:43:14.070201 16487 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790304178325258-13942-0/minicluster-data/ts-0/wal/instance:
uuid: "869f8d0294dd4190af070a5f009052c4"
format_stamp: "Formatted at 2026-09-25 02:43:14 on dist-test-slave-kznw"
server_key: "40f5a4aa101c7b424cb19bfb598df7d4"
server_key_iv: "1953d39e00a5c604c51156d6f58e5063"
server_key_version: "encryptionkey@0"
I20260925 02:43:14.073908 16487 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260925 02:43:14.076433 16502 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:14.077762 16487 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260925 02:43:14.077859 16487 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790304178325258-13942-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790304178325258-13942-0/minicluster-data/ts-0/wal
uuid: "869f8d0294dd4190af070a5f009052c4"
format_stamp: "Formatted at 2026-09-25 02:43:14 on dist-test-slave-kznw"
server_key: "40f5a4aa101c7b424cb19bfb598df7d4"
server_key_iv: "1953d39e00a5c604c51156d6f58e5063"
server_key_version: "encryptionkey@0"
I20260925 02:43:14.077945 16487 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790304178325258-13942-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790304178325258-13942-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790304178325258-13942-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:43:14.090801 16487 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:43:14.094180 16487 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:43:14.094343 16487 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:43:14.094949 16487 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:43:14.095767 16487 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:43:14.095818 16487 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:14.095858 16487 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:43:14.095878 16487 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:14.104496 16487 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.129:36853
I20260925 02:43:14.104558 16618 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.129:36853 every 8 connection(s)
I20260925 02:43:14.105397 16487 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790304178325258-13942-0/minicluster-data/ts-0/data/info.pb
I20260925 02:43:14.114717 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 16487
Sep 25 02:43:14 dist-test-slave-kznw krb5kdc[16377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304194, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.129@KRBTEST.COM for kudu/127.13.157.190@KRBTEST.COM
I20260925 02:43:14.114840 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790304178325258-13942-0/minicluster-data/ts-0/wal/instance
I20260925 02:43:14.115180 13942 external_mini_cluster.cc:1428] Setting key 6adf8e803a365168669bb1d173a7ddfe
I20260925 02:43:14.119573 16464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:14.107253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.129:57069 (local address 127.13.157.190:46743)
0925 02:43:14.107402 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:14.107407 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:14.108584 (+ 1177us) server_negotiation.cc:407] Connection header received
0925 02:43:14.109462 (+ 878us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:14.109465 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:14.109512 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:14.109584 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:14.110780 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:14.111409 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:14.112561 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:14.112809 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:14.112889 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:14.115614 (+ 2725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:14.115619 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:14.115622 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:43:14.115654 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:14.117290 (+ 1636us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:14.117880 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:14.117883 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:14.117884 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:14.117968 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:14.118345 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:14.118349 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:14.118351 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:14.118578 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:14.118708 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:14.119277 (+ 569us) server_negotiation.cc:299] Negotiation successful
0925 02:43:14.119420 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260925 02:43:14.120355 16621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:14.107734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:46743 (local address 127.13.157.129:57069)
0925 02:43:14.108432 (+ 698us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:14.108475 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:14.109285 (+ 810us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:14.109741 (+ 456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:14.109749 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:14.110101 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:14.110635 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:14.110649 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:14.111531 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:14.111534 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:14.112460 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:14.112471 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:14.112927 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:14.112930 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:14.113068 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:14.113641 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:14.113657 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:14.115392 (+ 1735us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:43:14.117413 (+ 2021us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:14.117419 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:14.117435 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:14.117771 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:14.118050 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:14.118053 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:14.118054 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:14.118252 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:14.118697 (+ 445us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:43:14.118702 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:43:14.118992 (+ 290us) client_negotiation.cc:769] Sending connection context
0925 02:43:14.119202 (+ 210us) client_negotiation.cc:240] Negotiation successful
0925 02:43:14.119468 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":115,"threads_started":1}
I20260925 02:43:14.121536 16619 heartbeater.cc:347] Connected to a master server at 127.13.157.190:46743
I20260925 02:43:14.121811 16619 heartbeater.cc:464] Registering TS with master...
I20260925 02:43:14.122411 16619 heartbeater.cc:511] Master 127.13.157.190:46743 requested a full tablet report, sending...
I20260925 02:43:14.124085 16423 ts_manager.cc:194] Registered new tserver with Master: 869f8d0294dd4190af070a5f009052c4 (127.13.157.129:36853)
WARNING: no policy specified for kudu/127.13.157.130@KRBTEST.COM; defaulting to no policy
I20260925 02:43:14.125262 16423 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.157.129@KRBTEST.COM'} at 127.13.157.129:57069
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.157.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.157.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.130.keytab.
Entry for principal kudu/127.13.157.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.130.keytab.
I20260925 02:43:14.141964 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790304178325258-13942-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790304178325258-13942-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790304178325258-13942-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790304178325258-13942-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.157.130:0
--local_ip_for_outbound_sockets=127.13.157.130
--webserver_interface=127.13.157.130
--webserver_port=0
--tserver_master_addrs=127.13.157.190:46743
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.130.keytab
--principal=kudu/127.13.157.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:44289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:43:14.252103 16625 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:43:14.252390 16625 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:43:14.252434 16625 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:43:14.256574 16625 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:43:14.256651 16625 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:43:14.256716 16625 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.157.130
I20260925 02:43:14.262025 16625 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:44289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790304178325258-13942-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790304178325258-13942-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.130.keytab
--principal=kudu/127.13.157.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.157.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790304178325258-13942-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.157.130
--webserver_port=0
--tserver_master_addrs=127.13.157.190:46743
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16625
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.157.130
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790304178325258-13942-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:43:14.263110 16625 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:43:14.264132 16625 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:43:14.273749 16633 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:14.273806 16630 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:43:14.273948 16625 server_base.cc:1034] running on GCE node
W20260925 02:43:14.273751 16631 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:43:14.274545 16625 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:43:14.275252 16625 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:43:14.276422 16625 hybrid_clock.cc:648] HybridClock initialized: now 1790304194276382 us; error 52 us; skew 500 ppm
Sep 25 02:43:14 dist-test-slave-kznw krb5kdc[16377](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304194, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:43:14.279163 16625 init.cc:377] Logged in from keytab as kudu/127.13.157.130@KRBTEST.COM (short username kudu)
I20260925 02:43:14.280117 16625 webserver.cc:460] Webserver started at http://127.13.157.130:37251/ using document root <none> and password file <none>
I20260925 02:43:14.280714 16625 fs_manager.cc:359] Metadata directory not provided
I20260925 02:43:14.280799 16625 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:43:14.280985 16625 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:43:14.283094 16625 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790304178325258-13942-0/minicluster-data/ts-1/data/instance:
uuid: "e54f34f82ec748088091205fd81efc1e"
format_stamp: "Formatted at 2026-09-25 02:43:14 on dist-test-slave-kznw"
server_key: "6ff3c3c8ef0f8e6bbfe80a24a8d0236f"
server_key_iv: "4d8a11a86a68cf40abbafb1254fea54c"
server_key_version: "encryptionkey@0"
I20260925 02:43:14.283618 16625 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790304178325258-13942-0/minicluster-data/ts-1/wal/instance:
uuid: "e54f34f82ec748088091205fd81efc1e"
format_stamp: "Formatted at 2026-09-25 02:43:14 on dist-test-slave-kznw"
server_key: "6ff3c3c8ef0f8e6bbfe80a24a8d0236f"
server_key_iv: "4d8a11a86a68cf40abbafb1254fea54c"
server_key_version: "encryptionkey@0"
I20260925 02:43:14.287319 16625 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260925 02:43:14.289884 16640 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:14.291461 16625 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 02:43:14.291572 16625 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790304178325258-13942-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790304178325258-13942-0/minicluster-data/ts-1/wal
uuid: "e54f34f82ec748088091205fd81efc1e"
format_stamp: "Formatted at 2026-09-25 02:43:14 on dist-test-slave-kznw"
server_key: "6ff3c3c8ef0f8e6bbfe80a24a8d0236f"
server_key_iv: "4d8a11a86a68cf40abbafb1254fea54c"
server_key_version: "encryptionkey@0"
I20260925 02:43:14.291675 16625 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790304178325258-13942-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790304178325258-13942-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790304178325258-13942-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:43:14.310571 16625 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:43:14.313850 16625 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:43:14.314018 16625 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:43:14.314620 16625 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:43:14.315507 16625 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:43:14.315567 16625 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:14.315608 16625 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:43:14.315630 16625 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:14.324400 16625 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.130:39271
I20260925 02:43:14.324424 16756 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.130:39271 every 8 connection(s)
I20260925 02:43:14.325320 16625 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790304178325258-13942-0/minicluster-data/ts-1/data/info.pb
I20260925 02:43:14.328433 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 16625
I20260925 02:43:14.328553 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790304178325258-13942-0/minicluster-data/ts-1/wal/instance
I20260925 02:43:14.328894 13942 external_mini_cluster.cc:1428] Setting key 45d9e9e2c525a44195c2200e82fa0945
Sep 25 02:43:14 dist-test-slave-kznw krb5kdc[16377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304194, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.130@KRBTEST.COM for kudu/127.13.157.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.157.131@KRBTEST.COM; defaulting to no policy
I20260925 02:43:14.340135 16464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:14.327208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.130:43613 (local address 127.13.157.190:46743)
0925 02:43:14.327380 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:14.327386 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:14.328579 (+ 1193us) server_negotiation.cc:407] Connection header received
0925 02:43:14.329665 (+ 1086us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:14.329669 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:14.329727 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:14.329797 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:14.331278 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:14.331861 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:14.333068 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:14.333298 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:14.333390 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:14.336087 (+ 2697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:14.336091 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:14.336094 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:43:14.336129 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:14.337787 (+ 1658us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:14.338393 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:14.338397 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:14.338399 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:14.338492 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:14.338891 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:14.338894 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:14.338896 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:14.339219 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:14.339305 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:14.339833 (+ 528us) server_negotiation.cc:299] Negotiation successful
0925 02:43:14.339957 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.157.131@KRBTEST.COM" created.
I20260925 02:43:14.341190 16759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:14.327615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:46743 (local address 127.13.157.130:43613)
0925 02:43:14.328390 (+ 775us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:14.328464 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:14.329451 (+ 987us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:14.329982 (+ 531us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:14.329989 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:14.330389 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:14.331090 (+ 701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:14.331124 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:14.331990 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:14.331993 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:14.332880 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:14.332890 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:14.333452 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:14.333456 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:14.333631 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:14.334238 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:14.334254 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:14.335896 (+ 1642us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:43:14.337914 (+ 2018us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:14.337919 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:14.337932 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:14.338241 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:14.338585 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:14.338587 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:14.338588 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:14.338727 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:14.339325 (+ 598us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:43:14.339331 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:43:14.339583 (+ 252us) client_negotiation.cc:769] Sending connection context
0925 02:43:14.339776 (+ 193us) client_negotiation.cc:240] Negotiation successful
0925 02:43:14.340085 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":528,"thread_start_us":122,"threads_started":1}
I20260925 02:43:14.342448 16757 heartbeater.cc:347] Connected to a master server at 127.13.157.190:46743
I20260925 02:43:14.342730 16757 heartbeater.cc:464] Registering TS with master...
I20260925 02:43:14.343340 16757 heartbeater.cc:511] Master 127.13.157.190:46743 requested a full tablet report, sending...
I20260925 02:43:14.344501 16423 ts_manager.cc:194] Registered new tserver with Master: e54f34f82ec748088091205fd81efc1e (127.13.157.130:39271)
I20260925 02:43:14.345088 16423 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.157.130@KRBTEST.COM'} at 127.13.157.130:43613
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.157.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.131.keytab.
Entry for principal kudu/127.13.157.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.131.keytab.
I20260925 02:43:14.357682 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790304178325258-13942-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790304178325258-13942-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790304178325258-13942-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790304178325258-13942-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.157.131:0
--local_ip_for_outbound_sockets=127.13.157.131
--webserver_interface=127.13.157.131
--webserver_port=0
--tserver_master_addrs=127.13.157.190:46743
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.131.keytab
--principal=kudu/127.13.157.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:44289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:43:14.466441 16762 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:43:14.466696 16762 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:43:14.466753 16762 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:43:14.471184 16762 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:43:14.471257 16762 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:43:14.471354 16762 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.157.131
I20260925 02:43:14.476815 16762 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:44289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790304178325258-13942-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790304178325258-13942-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.131.keytab
--principal=kudu/127.13.157.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.157.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790304178325258-13942-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.157.131
--webserver_port=0
--tserver_master_addrs=127.13.157.190:46743
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16762
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.157.131
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790304178325258-13942-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:43:14.477905 16762 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:43:14.478951 16762 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:43:14.488905 16770 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:14.488898 16767 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:14.488906 16768 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:43:14.489027 16762 server_base.cc:1034] running on GCE node
I20260925 02:43:14.489583 16762 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:43:14.490257 16762 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:43:14.491418 16762 hybrid_clock.cc:648] HybridClock initialized: now 1790304194491397 us; error 30 us; skew 500 ppm
Sep 25 02:43:14 dist-test-slave-kznw krb5kdc[16377](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304194, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:43:14.494130 16762 init.cc:377] Logged in from keytab as kudu/127.13.157.131@KRBTEST.COM (short username kudu)
I20260925 02:43:14.495048 16762 webserver.cc:460] Webserver started at http://127.13.157.131:40979/ using document root <none> and password file <none>
I20260925 02:43:14.495626 16762 fs_manager.cc:359] Metadata directory not provided
I20260925 02:43:14.495687 16762 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:43:14.495859 16762 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:43:14.497975 16762 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790304178325258-13942-0/minicluster-data/ts-2/data/instance:
uuid: "6edee51221074d718a10520b05038323"
format_stamp: "Formatted at 2026-09-25 02:43:14 on dist-test-slave-kznw"
server_key: "46b680f41e4e39b92dc009866d5956e9"
server_key_iv: "f16dfbbdc98465841c042c64079dfb7a"
server_key_version: "encryptionkey@0"
I20260925 02:43:14.498505 16762 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790304178325258-13942-0/minicluster-data/ts-2/wal/instance:
uuid: "6edee51221074d718a10520b05038323"
format_stamp: "Formatted at 2026-09-25 02:43:14 on dist-test-slave-kznw"
server_key: "46b680f41e4e39b92dc009866d5956e9"
server_key_iv: "f16dfbbdc98465841c042c64079dfb7a"
server_key_version: "encryptionkey@0"
I20260925 02:43:14.502197 16762 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260925 02:43:14.504535 16777 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:14.505841 16762 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260925 02:43:14.505964 16762 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790304178325258-13942-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790304178325258-13942-0/minicluster-data/ts-2/wal
uuid: "6edee51221074d718a10520b05038323"
format_stamp: "Formatted at 2026-09-25 02:43:14 on dist-test-slave-kznw"
server_key: "46b680f41e4e39b92dc009866d5956e9"
server_key_iv: "f16dfbbdc98465841c042c64079dfb7a"
server_key_version: "encryptionkey@0"
I20260925 02:43:14.506114 16762 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790304178325258-13942-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790304178325258-13942-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790304178325258-13942-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:43:14.539309 16762 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:43:14.542647 16762 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:43:14.542835 16762 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:43:14.543504 16762 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:43:14.544392 16762 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:43:14.544440 16762 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:14.544482 16762 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:43:14.544509 16762 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:14.553084 16762 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.131:37805
I20260925 02:43:14.553107 16894 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.131:37805 every 8 connection(s)
I20260925 02:43:14.554064 16762 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790304178325258-13942-0/minicluster-data/ts-2/data/info.pb
Sep 25 02:43:14 dist-test-slave-kznw krb5kdc[16377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304194, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.131@KRBTEST.COM for kudu/127.13.157.190@KRBTEST.COM
I20260925 02:43:14.564203 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 16762
I20260925 02:43:14.564364 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790304178325258-13942-0/minicluster-data/ts-2/wal/instance
I20260925 02:43:14.564970 13942 external_mini_cluster.cc:1428] Setting key 6c9caade3464139307ea23ac47737cc3
I20260925 02:43:14.567718 16464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:14.555991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.131:54905 (local address 127.13.157.190:46743)
0925 02:43:14.556136 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:14.556140 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:14.557263 (+ 1123us) server_negotiation.cc:407] Connection header received
0925 02:43:14.558122 (+ 859us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:14.558125 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:14.558178 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:14.558242 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:14.559531 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:14.560116 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:14.561263 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:14.561480 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:14.561566 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:14.564139 (+ 2573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:14.564143 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:14.564146 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:43:14.564178 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:14.565702 (+ 1524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:14.566190 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:14.566193 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:14.566195 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:14.566274 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:14.566591 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:14.566595 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:14.566596 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:14.566779 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:14.566860 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:14.567425 (+ 565us) server_negotiation.cc:299] Negotiation successful
0925 02:43:14.567557 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260925 02:43:14.568570 16897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:14.556412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:46743 (local address 127.13.157.131:54905)
0925 02:43:14.557113 (+ 701us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:14.557157 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:14.557927 (+ 770us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:14.558401 (+ 474us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:14.558408 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:14.558790 (+ 382us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:14.559341 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:14.559356 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:14.560230 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:14.560232 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:14.561146 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:14.561156 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:14.561642 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:14.561645 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:14.561787 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:14.562339 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:14.562354 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:14.563947 (+ 1593us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:43:14.565818 (+ 1871us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:14.565824 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:14.565839 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:14.566097 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:14.566356 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:14.566359 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:14.566362 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:14.566495 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:14.566885 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:43:14.566892 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:43:14.567193 (+ 301us) client_negotiation.cc:769] Sending connection context
0925 02:43:14.567368 (+ 175us) client_negotiation.cc:240] Negotiation successful
0925 02:43:14.567700 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":121,"threads_started":1}
I20260925 02:43:14.569742 16895 heartbeater.cc:347] Connected to a master server at 127.13.157.190:46743
I20260925 02:43:14.569993 16895 heartbeater.cc:464] Registering TS with master...
I20260925 02:43:14.570503 16895 heartbeater.cc:511] Master 127.13.157.190:46743 requested a full tablet report, sending...
I20260925 02:43:14.571660 16423 ts_manager.cc:194] Registered new tserver with Master: 6edee51221074d718a10520b05038323 (127.13.157.131:37805)
I20260925 02:43:14.572253 16423 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.157.131@KRBTEST.COM'} at 127.13.157.131:54905
I20260925 02:43:14.579329 13942 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260925 02:43:14.593568 16464 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:43:14.592224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53400 (local address 127.13.157.190:46743)
0925 02:43:14.592381 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:14.592385 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:14.592403 (+ 18us) server_negotiation.cc:407] Connection header received
0925 02:43:14.592546 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:14.592549 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:14.592595 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:14.592664 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:14.593439 (+ 775us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53400: BlockingRecv error: recv got EOF from 127.0.0.1:53400 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
I20260925 02:43:14.594730 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 16487
I20260925 02:43:14.600050 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 16625
I20260925 02:43:14.605069 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 16762
I20260925 02:43:14.610150 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 16390
2026-09-25T02:43:14Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2032 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790304178325258-13942-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 02:43:14 dist-test-slave-kznw krb5kdc[16907](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 02:43:14 dist-test-slave-kznw krb5kdc[16907](info): set up 2 sockets
Sep 25 02:43:14 dist-test-slave-kznw krb5kdc[16907](info): commencing operation
krb5kdc: starting...
W20260925 02:43:15.570307 13942 mini_kdc.cc:121] Time spent starting KDC: real 0.939s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790304178325258-13942-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790304178325258-13942-0/krb5kdc/test-user.keytab.
Sep 25 02:43:15 dist-test-slave-kznw krb5kdc[16907](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304195, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T02:43:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T02:43:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.157.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.157.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.157.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab.
Entry for principal kudu/127.13.157.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab.
I20260925 02:43:15.687994 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790304178325258-13942-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790304178325258-13942-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790304178325258-13942-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.157.190:45293
--webserver_interface=127.13.157.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab
--principal=kudu/127.13.157.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:46445
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.157.190:45293
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:43:15.796548 16920 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:43:15.796814 16920 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:43:15.796855 16920 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:43:15.800902 16920 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:43:15.800956 16920 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:43:15.800974 16920 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:43:15.800989 16920 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:43:15.801003 16920 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:43:15.806254 16920 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:46445
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790304178325258-13942-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790304178325258-13942-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.157.190:45293
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab
--principal=kudu/127.13.157.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.157.190:45293
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.157.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16920
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790304178325258-13942-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:43:15.807382 16920 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:43:15.808413 16920 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:43:15.816769 16928 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:43:15.816906 16920 server_base.cc:1034] running on GCE node
W20260925 02:43:15.816846 16925 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:15.816831 16926 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:43:15.817744 16920 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:43:15.818885 16920 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:43:15.820079 16920 hybrid_clock.cc:648] HybridClock initialized: now 1790304195820064 us; error 26 us; skew 500 ppm
Sep 25 02:43:15 dist-test-slave-kznw krb5kdc[16907](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304195, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:43:15.822846 16920 init.cc:377] Logged in from keytab as kudu/127.13.157.190@KRBTEST.COM (short username kudu)
I20260925 02:43:15.823791 16920 webserver.cc:460] Webserver started at http://127.13.157.190:39027/ using document root <none> and password file <none>
I20260925 02:43:15.824334 16920 fs_manager.cc:359] Metadata directory not provided
I20260925 02:43:15.824395 16920 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:43:15.824563 16920 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:43:15.826591 16920 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790304178325258-13942-0/minicluster-data/master-0/data/instance:
uuid: "d4a0be5abef0466ea557e46694e65067"
format_stamp: "Formatted at 2026-09-25 02:43:15 on dist-test-slave-kznw"
server_key: "dfc79da23957108f10c05e8fcb874113"
server_key_iv: "347b3464d7fdd6d455379ae6ecc51cc4"
server_key_version: "encryptionkey@0"
I20260925 02:43:15.827171 16920 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790304178325258-13942-0/minicluster-data/master-0/wal/instance:
uuid: "d4a0be5abef0466ea557e46694e65067"
format_stamp: "Formatted at 2026-09-25 02:43:15 on dist-test-slave-kznw"
server_key: "dfc79da23957108f10c05e8fcb874113"
server_key_iv: "347b3464d7fdd6d455379ae6ecc51cc4"
server_key_version: "encryptionkey@0"
I20260925 02:43:15.830891 16920 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260925 02:43:15.833561 16935 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:15.834892 16920 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260925 02:43:15.835062 16920 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790304178325258-13942-0/minicluster-data/master-0/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790304178325258-13942-0/minicluster-data/master-0/wal
uuid: "d4a0be5abef0466ea557e46694e65067"
format_stamp: "Formatted at 2026-09-25 02:43:15 on dist-test-slave-kznw"
server_key: "dfc79da23957108f10c05e8fcb874113"
server_key_iv: "347b3464d7fdd6d455379ae6ecc51cc4"
server_key_version: "encryptionkey@0"
I20260925 02:43:15.835179 16920 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790304178325258-13942-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790304178325258-13942-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790304178325258-13942-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:43:15.858410 16920 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:43:15.861800 16920 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:43:15.861989 16920 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:43:15.868885 16920 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.190:45293
I20260925 02:43:15.869185 16988 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.190:45293 every 8 connection(s)
I20260925 02:43:15.869863 16920 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
I20260925 02:43:15.872727 16989 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:43:15.873996 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 16920
I20260925 02:43:15.874118 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790304178325258-13942-0/minicluster-data/master-0/wal/instance
I20260925 02:43:15.874398 13942 external_mini_cluster.cc:1428] Setting key f5edb788137d3aa53aea74a5e1ad6b39
I20260925 02:43:15.879271 16989 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d4a0be5abef0466ea557e46694e65067: Bootstrap starting.
I20260925 02:43:15.881515 16989 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d4a0be5abef0466ea557e46694e65067: Neither blocks nor log segments found. Creating new log.
I20260925 02:43:15.882331 16989 log.cc:824] T 00000000000000000000000000000000 P d4a0be5abef0466ea557e46694e65067: Log is configured to *not* fsync() on all Append() calls
Sep 25 02:43:15 dist-test-slave-kznw krb5kdc[16907](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304195, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.157.190@KRBTEST.COM
I20260925 02:43:15.884400 16989 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d4a0be5abef0466ea557e46694e65067: No bootstrap required, opened a new log
I20260925 02:43:15.888099 16989 raft_consensus.cc:348] T 00000000000000000000000000000000 P d4a0be5abef0466ea557e46694e65067 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4a0be5abef0466ea557e46694e65067" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 45293 } }
I20260925 02:43:15.888332 16989 raft_consensus.cc:374] T 00000000000000000000000000000000 P d4a0be5abef0466ea557e46694e65067 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:43:15.888366 16989 raft_consensus.cc:729] T 00000000000000000000000000000000 P d4a0be5abef0466ea557e46694e65067 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4a0be5abef0466ea557e46694e65067, State: Initialized, Role: FOLLOWER
I20260925 02:43:15.888805 16992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:15.875876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34114 (local address 127.13.157.190:45293)
0925 02:43:15.876400 (+ 524us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:15.876422 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:15.876474 (+ 52us) server_negotiation.cc:407] Connection header received
0925 02:43:15.877374 (+ 900us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:15.877396 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:15.877772 (+ 376us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:15.878090 (+ 318us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:15.878961 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:15.879963 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:15.880830 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:15.881301 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:15.881449 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:15.883930 (+ 2481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:15.883934 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:15.883947 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:43:15.883974 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:15.885685 (+ 1711us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:15.886155 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:15.886160 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:15.886164 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:15.886303 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:15.886607 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:15.886611 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:15.886613 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:15.887121 (+ 508us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:15.887349 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:15.887669 (+ 320us) server_negotiation.cc:299] Negotiation successful
0925 02:43:15.887971 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":149,"threads_started":1}
I20260925 02:43:15.888819 16989 consensus_queue.cc:260] T 00000000000000000000000000000000 P d4a0be5abef0466ea557e46694e65067 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4a0be5abef0466ea557e46694e65067" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 45293 } }
I20260925 02:43:15.888978 16989 raft_consensus.cc:388] T 00000000000000000000000000000000 P d4a0be5abef0466ea557e46694e65067 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 02:43:15.889083 16989 raft_consensus.cc:482] T 00000000000000000000000000000000 P d4a0be5abef0466ea557e46694e65067 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 02:43:15.889189 16989 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d4a0be5abef0466ea557e46694e65067 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:43:15.890094 16989 raft_consensus.cc:504] T 00000000000000000000000000000000 P d4a0be5abef0466ea557e46694e65067 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4a0be5abef0466ea557e46694e65067" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 45293 } }
I20260925 02:43:15.890414 16989 leader_election.cc:302] T 00000000000000000000000000000000 P d4a0be5abef0466ea557e46694e65067 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d4a0be5abef0466ea557e46694e65067; no voters:
I20260925 02:43:15.890724 16989 leader_election.cc:288] T 00000000000000000000000000000000 P d4a0be5abef0466ea557e46694e65067 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 02:43:15.890821 16994 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d4a0be5abef0466ea557e46694e65067 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:43:15.891114 16994 raft_consensus.cc:686] T 00000000000000000000000000000000 P d4a0be5abef0466ea557e46694e65067 [term 1 LEADER]: Becoming Leader. State: Replica: d4a0be5abef0466ea557e46694e65067, State: Running, Role: LEADER
I20260925 02:43:15.891474 16994 consensus_queue.cc:237] T 00000000000000000000000000000000 P d4a0be5abef0466ea557e46694e65067 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4a0be5abef0466ea557e46694e65067" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 45293 } }
I20260925 02:43:15.891817 13942 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 02:43:15.892108 16989 sys_catalog.cc:564] T 00000000000000000000000000000000 P d4a0be5abef0466ea557e46694e65067 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:43:15.893224 16996 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4a0be5abef0466ea557e46694e65067 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d4a0be5abef0466ea557e46694e65067. Latest consensus state: current_term: 1 leader_uuid: "d4a0be5abef0466ea557e46694e65067" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4a0be5abef0466ea557e46694e65067" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 45293 } } }
I20260925 02:43:15.893225 16995 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4a0be5abef0466ea557e46694e65067 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d4a0be5abef0466ea557e46694e65067" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4a0be5abef0466ea557e46694e65067" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 45293 } } }
I20260925 02:43:15.893378 16995 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4a0be5abef0466ea557e46694e65067 [sys.catalog]: This master's current role is: LEADER
I20260925 02:43:15.893378 16996 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4a0be5abef0466ea557e46694e65067 [sys.catalog]: This master's current role is: LEADER
W20260925 02:43:15.897136 17011 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d4a0be5abef0466ea557e46694e65067: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 02:43:15.897208 17011 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 02:43:15.897292 17003 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:43:15.898209 17003 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:43:15.904310 17003 catalog_manager.cc:1347] Generated new cluster ID: d658bfc1a33d499fa85aa68946de955c
I20260925 02:43:15.904379 17003 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 02:43:15.913990 16992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:15.908265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34126 (local address 127.13.157.190:45293)
0925 02:43:15.908568 (+ 303us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:15.908574 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:15.908617 (+ 43us) server_negotiation.cc:407] Connection header received
0925 02:43:15.908780 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:15.908784 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:15.908837 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:15.908928 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:15.909566 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:15.910367 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:15.911606 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:15.911964 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:15.912056 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:15.912956 (+ 900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:15.912960 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:15.912963 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:43:15.913548 (+ 585us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:15.913583 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:43:15.913604 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:15.913682 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:15.913780 (+ 98us) server_negotiation.cc:299] Negotiation successful
0925 02:43:15.913838 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":213}
I20260925 02:43:15.939142 17003 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 02:43:15.940150 17003 catalog_manager.cc:1504] Loading token signing keys...
I20260925 02:43:15.945906 17003 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d4a0be5abef0466ea557e46694e65067: Generated new TSK 0
I20260925 02:43:15.946743 17003 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 02:43:15.956692 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 16920
2026-09-25T02:43:15Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1345 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790304178325258-13942-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 02:43:16 dist-test-slave-kznw krb5kdc[17023](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 02:43:16 dist-test-slave-kznw krb5kdc[17023](info): set up 2 sockets
Sep 25 02:43:16 dist-test-slave-kznw krb5kdc[17023](info): commencing operation
krb5kdc: starting...
W20260925 02:43:16.910519 13942 mini_kdc.cc:121] Time spent starting KDC: real 0.934s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790304178325258-13942-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790304178325258-13942-0/krb5kdc/test-user.keytab.
Sep 25 02:43:16 dist-test-slave-kznw krb5kdc[17023](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304196, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T02:43:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T02:43:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.157.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.157.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.157.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab.
Entry for principal kudu/127.13.157.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab.
I20260925 02:43:17.027707 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790304178325258-13942-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790304178325258-13942-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790304178325258-13942-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.157.190:42067
--webserver_interface=127.13.157.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab
--principal=kudu/127.13.157.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:38289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.157.190:42067
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:43:17.137324 17036 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:43:17.137620 17036 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:43:17.137682 17036 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:43:17.141824 17036 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:43:17.141887 17036 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:43:17.141916 17036 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:43:17.141935 17036 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:43:17.141956 17036 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:43:17.147367 17036 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:38289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790304178325258-13942-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790304178325258-13942-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.157.190:42067
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab
--principal=kudu/127.13.157.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.157.190:42067
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.157.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17036
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790304178325258-13942-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:43:17.148427 17036 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:43:17.149482 17036 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 02:43:17.158056 17036 server_base.cc:1034] running on GCE node
W20260925 02:43:17.158005 17044 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:17.157990 17041 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:17.158008 17042 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:43:17.158891 17036 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:43:17.160143 17036 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:43:17.161317 17036 hybrid_clock.cc:648] HybridClock initialized: now 1790304197161299 us; error 32 us; skew 500 ppm
Sep 25 02:43:17 dist-test-slave-kznw krb5kdc[17023](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304197, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:43:17.164318 17036 init.cc:377] Logged in from keytab as kudu/127.13.157.190@KRBTEST.COM (short username kudu)
I20260925 02:43:17.165284 17036 webserver.cc:460] Webserver started at http://127.13.157.190:36741/ using document root <none> and password file <none>
I20260925 02:43:17.165829 17036 fs_manager.cc:359] Metadata directory not provided
I20260925 02:43:17.165899 17036 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:43:17.166124 17036 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:43:17.168313 17036 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790304178325258-13942-0/minicluster-data/master-0/data/instance:
uuid: "8a4332e1f9924adf8d61701c719951ea"
format_stamp: "Formatted at 2026-09-25 02:43:17 on dist-test-slave-kznw"
server_key: "706f0081a2e072a7279e370bb806723b"
server_key_iv: "9b834dfad1b2a49f97bdca54cf93495b"
server_key_version: "encryptionkey@0"
I20260925 02:43:17.168877 17036 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790304178325258-13942-0/minicluster-data/master-0/wal/instance:
uuid: "8a4332e1f9924adf8d61701c719951ea"
format_stamp: "Formatted at 2026-09-25 02:43:17 on dist-test-slave-kznw"
server_key: "706f0081a2e072a7279e370bb806723b"
server_key_iv: "9b834dfad1b2a49f97bdca54cf93495b"
server_key_version: "encryptionkey@0"
I20260925 02:43:17.172693 17036 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260925 02:43:17.175202 17051 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:17.176499 17036 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260925 02:43:17.176627 17036 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790304178325258-13942-0/minicluster-data/master-0/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790304178325258-13942-0/minicluster-data/master-0/wal
uuid: "8a4332e1f9924adf8d61701c719951ea"
format_stamp: "Formatted at 2026-09-25 02:43:17 on dist-test-slave-kznw"
server_key: "706f0081a2e072a7279e370bb806723b"
server_key_iv: "9b834dfad1b2a49f97bdca54cf93495b"
server_key_version: "encryptionkey@0"
I20260925 02:43:17.176738 17036 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790304178325258-13942-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790304178325258-13942-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790304178325258-13942-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:43:17.199710 17036 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:43:17.202870 17036 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:43:17.203105 17036 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:43:17.209872 17036 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.190:42067
I20260925 02:43:17.210117 17105 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.190:42067 every 8 connection(s)
I20260925 02:43:17.210871 17036 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
I20260925 02:43:17.213667 17106 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:43:17.213722 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 17036
I20260925 02:43:17.213841 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790304178325258-13942-0/minicluster-data/master-0/wal/instance
I20260925 02:43:17.214107 13942 external_mini_cluster.cc:1428] Setting key 5a452aab88ca588d0db41d21922c5811
I20260925 02:43:17.220628 17106 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a4332e1f9924adf8d61701c719951ea: Bootstrap starting.
I20260925 02:43:17.223153 17106 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8a4332e1f9924adf8d61701c719951ea: Neither blocks nor log segments found. Creating new log.
Sep 25 02:43:17 dist-test-slave-kznw krb5kdc[17023](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304196, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.157.190@KRBTEST.COM
I20260925 02:43:17.223919 17106 log.cc:824] T 00000000000000000000000000000000 P 8a4332e1f9924adf8d61701c719951ea: Log is configured to *not* fsync() on all Append() calls
I20260925 02:43:17.226119 17106 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a4332e1f9924adf8d61701c719951ea: No bootstrap required, opened a new log
I20260925 02:43:17.228650 17109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:17.215541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59122 (local address 127.13.157.190:42067)
0925 02:43:17.215973 (+ 432us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:17.215993 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:17.216037 (+ 44us) server_negotiation.cc:407] Connection header received
0925 02:43:17.216860 (+ 823us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:17.216887 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:17.217313 (+ 426us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:17.217692 (+ 379us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:17.218327 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:17.219890 (+ 1563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:17.220726 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:17.221381 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:17.221533 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:17.223921 (+ 2388us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:17.223927 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:17.223944 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:43:17.223978 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:17.225643 (+ 1665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:17.226123 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:17.226127 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:17.226132 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:17.226272 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:17.226592 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:17.226595 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:17.226596 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:17.227060 (+ 464us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:17.227237 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:17.227603 (+ 366us) server_negotiation.cc:299] Negotiation successful
0925 02:43:17.227910 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":285,"thread_start_us":123,"threads_started":1}
I20260925 02:43:17.229950 17106 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8a4332e1f9924adf8d61701c719951ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a4332e1f9924adf8d61701c719951ea" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 42067 } }
I20260925 02:43:17.230181 17106 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8a4332e1f9924adf8d61701c719951ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:43:17.230237 17106 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8a4332e1f9924adf8d61701c719951ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a4332e1f9924adf8d61701c719951ea, State: Initialized, Role: FOLLOWER
I20260925 02:43:17.230659 17106 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8a4332e1f9924adf8d61701c719951ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a4332e1f9924adf8d61701c719951ea" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 42067 } }
I20260925 02:43:17.230789 17106 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8a4332e1f9924adf8d61701c719951ea [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 02:43:17.230855 17106 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8a4332e1f9924adf8d61701c719951ea [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 02:43:17.230976 17106 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8a4332e1f9924adf8d61701c719951ea [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:43:17.231870 17106 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8a4332e1f9924adf8d61701c719951ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a4332e1f9924adf8d61701c719951ea" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 42067 } }
I20260925 02:43:17.232191 17106 leader_election.cc:302] T 00000000000000000000000000000000 P 8a4332e1f9924adf8d61701c719951ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8a4332e1f9924adf8d61701c719951ea; no voters:
I20260925 02:43:17.232523 17106 leader_election.cc:288] T 00000000000000000000000000000000 P 8a4332e1f9924adf8d61701c719951ea [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 02:43:17.232700 17111 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8a4332e1f9924adf8d61701c719951ea [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:43:17.232897 17111 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8a4332e1f9924adf8d61701c719951ea [term 1 LEADER]: Becoming Leader. State: Replica: 8a4332e1f9924adf8d61701c719951ea, State: Running, Role: LEADER
I20260925 02:43:17.233237 17111 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8a4332e1f9924adf8d61701c719951ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a4332e1f9924adf8d61701c719951ea" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 42067 } }
I20260925 02:43:17.233735 17106 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8a4332e1f9924adf8d61701c719951ea [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:43:17.234748 17113 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a4332e1f9924adf8d61701c719951ea [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8a4332e1f9924adf8d61701c719951ea. Latest consensus state: current_term: 1 leader_uuid: "8a4332e1f9924adf8d61701c719951ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a4332e1f9924adf8d61701c719951ea" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 42067 } } }
I20260925 02:43:17.234876 17113 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a4332e1f9924adf8d61701c719951ea [sys.catalog]: This master's current role is: LEADER
I20260925 02:43:17.235289 17120 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:43:17.236029 17112 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a4332e1f9924adf8d61701c719951ea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8a4332e1f9924adf8d61701c719951ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a4332e1f9924adf8d61701c719951ea" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 42067 } } }
I20260925 02:43:17.236123 17112 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a4332e1f9924adf8d61701c719951ea [sys.catalog]: This master's current role is: LEADER
I20260925 02:43:17.237967 17120 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:43:17.243681 17120 catalog_manager.cc:1347] Generated new cluster ID: 6c0b1a54ff5c47e7976a27b995bf6ac4
I20260925 02:43:17.243755 17120 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 02:43:17.266800 17120 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 02:43:17.268172 17120 catalog_manager.cc:1504] Loading token signing keys...
I20260925 02:43:17.285269 17120 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8a4332e1f9924adf8d61701c719951ea: Generated new TSK 0
I20260925 02:43:17.286242 17120 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.157.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.157.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.157.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.129.keytab.
Entry for principal kudu/127.13.157.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.129.keytab.
I20260925 02:43:17.322655 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790304178325258-13942-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790304178325258-13942-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790304178325258-13942-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790304178325258-13942-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.157.129:0
--local_ip_for_outbound_sockets=127.13.157.129
--webserver_interface=127.13.157.129
--webserver_port=0
--tserver_master_addrs=127.13.157.190:42067
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.129.keytab
--principal=kudu/127.13.157.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:38289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:43:17.431522 17132 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:43:17.431816 17132 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:43:17.431886 17132 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:43:17.436085 17132 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:43:17.436149 17132 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:43:17.436233 17132 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.157.129
I20260925 02:43:17.441439 17132 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:38289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790304178325258-13942-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790304178325258-13942-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.129.keytab
--principal=kudu/127.13.157.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.157.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790304178325258-13942-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.157.129
--webserver_port=0
--tserver_master_addrs=127.13.157.190:42067
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17132
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.157.129
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790304178325258-13942-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:43:17.442468 17132 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:43:17.443487 17132 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:43:17.452908 17138 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:17.452855 17137 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:17.452845 17140 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:43:17.453004 17132 server_base.cc:1034] running on GCE node
I20260925 02:43:17.453519 17132 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:43:17.454149 17132 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:43:17.455327 17132 hybrid_clock.cc:648] HybridClock initialized: now 1790304197455300 us; error 34 us; skew 500 ppm
Sep 25 02:43:17 dist-test-slave-kznw krb5kdc[17023](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304197, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:43:17.458004 17132 init.cc:377] Logged in from keytab as kudu/127.13.157.129@KRBTEST.COM (short username kudu)
I20260925 02:43:17.458945 17132 webserver.cc:460] Webserver started at http://127.13.157.129:32895/ using document root <none> and password file <none>
I20260925 02:43:17.459494 17132 fs_manager.cc:359] Metadata directory not provided
I20260925 02:43:17.459570 17132 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:43:17.459784 17132 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:43:17.461766 17132 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790304178325258-13942-0/minicluster-data/ts-0/data/instance:
uuid: "a4ca6fbca0f440fca04bcdd2c7d93040"
format_stamp: "Formatted at 2026-09-25 02:43:17 on dist-test-slave-kznw"
server_key: "4ccf7fd0243534f98a83fe467dabaa3d"
server_key_iv: "ba65e4f22067cd022aedc56da778f40e"
server_key_version: "encryptionkey@0"
I20260925 02:43:17.462318 17132 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790304178325258-13942-0/minicluster-data/ts-0/wal/instance:
uuid: "a4ca6fbca0f440fca04bcdd2c7d93040"
format_stamp: "Formatted at 2026-09-25 02:43:17 on dist-test-slave-kznw"
server_key: "4ccf7fd0243534f98a83fe467dabaa3d"
server_key_iv: "ba65e4f22067cd022aedc56da778f40e"
server_key_version: "encryptionkey@0"
I20260925 02:43:17.465902 17132 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260925 02:43:17.468238 17147 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:17.469435 17132 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260925 02:43:17.469527 17132 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790304178325258-13942-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790304178325258-13942-0/minicluster-data/ts-0/wal
uuid: "a4ca6fbca0f440fca04bcdd2c7d93040"
format_stamp: "Formatted at 2026-09-25 02:43:17 on dist-test-slave-kznw"
server_key: "4ccf7fd0243534f98a83fe467dabaa3d"
server_key_iv: "ba65e4f22067cd022aedc56da778f40e"
server_key_version: "encryptionkey@0"
I20260925 02:43:17.469609 17132 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790304178325258-13942-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790304178325258-13942-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790304178325258-13942-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:43:17.479764 17132 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:43:17.482820 17132 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:43:17.483057 17132 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:43:17.483683 17132 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:43:17.484530 17132 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:43:17.484581 17132 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:17.484624 17132 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:43:17.484644 17132 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:17.494163 17132 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.129:45023
I20260925 02:43:17.494262 17261 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.129:45023 every 8 connection(s)
I20260925 02:43:17.495187 17132 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790304178325258-13942-0/minicluster-data/ts-0/data/info.pb
I20260925 02:43:17.498674 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 17132
I20260925 02:43:17.498773 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790304178325258-13942-0/minicluster-data/ts-0/wal/instance
I20260925 02:43:17.499079 13942 external_mini_cluster.cc:1428] Setting key 66e555fa0e1f1ed3a0a9d46c57818017
Sep 25 02:43:17 dist-test-slave-kznw krb5kdc[17023](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304197, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.129@KRBTEST.COM for kudu/127.13.157.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.157.130@KRBTEST.COM; defaulting to no policy
I20260925 02:43:17.509796 17109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:17.497001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.129:42085 (local address 127.13.157.190:42067)
0925 02:43:17.497180 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:17.497185 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:17.498075 (+ 890us) server_negotiation.cc:407] Connection header received
0925 02:43:17.499584 (+ 1509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:17.499588 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:17.499644 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:17.499710 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:17.500955 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:17.501481 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:17.503051 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:17.503285 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:17.503373 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:17.506237 (+ 2864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:17.506240 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:17.506242 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:43:17.506271 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:17.507594 (+ 1323us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:17.508163 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:17.508166 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:17.508168 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:17.508263 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:17.508620 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:17.508622 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:17.508624 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:17.508840 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:17.508919 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:17.509554 (+ 635us) server_negotiation.cc:299] Negotiation successful
0925 02:43:17.509672 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260925 02:43:17.510615 17264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:17.497434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:42067 (local address 127.13.157.129:42085)
0925 02:43:17.497910 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:17.497960 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:17.499363 (+ 1403us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:17.499906 (+ 543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:17.499913 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:17.500316 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:17.500820 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:17.500834 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:17.501596 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:17.501599 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:17.502936 (+ 1337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:17.502947 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:17.503434 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:17.503437 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:17.503627 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:17.504281 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:17.504302 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:17.506045 (+ 1743us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:43:17.507713 (+ 1668us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:17.507720 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:17.507738 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:17.508065 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:17.508345 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:17.508349 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:17.508351 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:17.508530 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:17.508938 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:43:17.508944 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:43:17.509281 (+ 337us) client_negotiation.cc:769] Sending connection context
0925 02:43:17.509454 (+ 173us) client_negotiation.cc:240] Negotiation successful
0925 02:43:17.509731 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":125,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.157.130@KRBTEST.COM" created.
I20260925 02:43:17.511929 17262 heartbeater.cc:347] Connected to a master server at 127.13.157.190:42067
I20260925 02:43:17.512171 17262 heartbeater.cc:464] Registering TS with master...
I20260925 02:43:17.512686 17262 heartbeater.cc:511] Master 127.13.157.190:42067 requested a full tablet report, sending...
I20260925 02:43:17.514264 17069 ts_manager.cc:194] Registered new tserver with Master: a4ca6fbca0f440fca04bcdd2c7d93040 (127.13.157.129:45023)
I20260925 02:43:17.515887 17069 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.157.129@KRBTEST.COM'} at 127.13.157.129:42085
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.157.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.130.keytab.
Entry for principal kudu/127.13.157.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.130.keytab.
I20260925 02:43:17.526345 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790304178325258-13942-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790304178325258-13942-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790304178325258-13942-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790304178325258-13942-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.157.130:0
--local_ip_for_outbound_sockets=127.13.157.130
--webserver_interface=127.13.157.130
--webserver_port=0
--tserver_master_addrs=127.13.157.190:42067
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.130.keytab
--principal=kudu/127.13.157.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:38289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:43:17.636416 17267 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:43:17.636682 17267 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:43:17.636725 17267 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:43:17.640805 17267 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:43:17.640872 17267 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:43:17.640935 17267 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.157.130
I20260925 02:43:17.646409 17267 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:38289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790304178325258-13942-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790304178325258-13942-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.130.keytab
--principal=kudu/127.13.157.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.157.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790304178325258-13942-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.157.130
--webserver_port=0
--tserver_master_addrs=127.13.157.190:42067
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17267
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.157.130
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790304178325258-13942-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:43:17.647533 17267 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:43:17.648598 17267 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 02:43:17.658427 17267 server_base.cc:1034] running on GCE node
W20260925 02:43:17.658459 17273 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:17.658376 17275 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:17.658386 17272 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:43:17.659097 17267 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:43:17.659914 17267 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:43:17.661113 17267 hybrid_clock.cc:648] HybridClock initialized: now 1790304197661078 us; error 43 us; skew 500 ppm
Sep 25 02:43:17 dist-test-slave-kznw krb5kdc[17023](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304197, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:43:17.663939 17267 init.cc:377] Logged in from keytab as kudu/127.13.157.130@KRBTEST.COM (short username kudu)
I20260925 02:43:17.664788 17267 webserver.cc:460] Webserver started at http://127.13.157.130:42225/ using document root <none> and password file <none>
I20260925 02:43:17.665310 17267 fs_manager.cc:359] Metadata directory not provided
I20260925 02:43:17.665369 17267 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:43:17.665534 17267 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:43:17.667650 17267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790304178325258-13942-0/minicluster-data/ts-1/data/instance:
uuid: "2f22299df6604012a7aafb427f1f8ba8"
format_stamp: "Formatted at 2026-09-25 02:43:17 on dist-test-slave-kznw"
server_key: "bcc4b5941d75fb7e0109cf8fc72a6dab"
server_key_iv: "03dac388429eabfee895eed906d45b40"
server_key_version: "encryptionkey@0"
I20260925 02:43:17.668171 17267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790304178325258-13942-0/minicluster-data/ts-1/wal/instance:
uuid: "2f22299df6604012a7aafb427f1f8ba8"
format_stamp: "Formatted at 2026-09-25 02:43:17 on dist-test-slave-kznw"
server_key: "bcc4b5941d75fb7e0109cf8fc72a6dab"
server_key_iv: "03dac388429eabfee895eed906d45b40"
server_key_version: "encryptionkey@0"
I20260925 02:43:17.671864 17267 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260925 02:43:17.674322 17282 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:17.675640 17267 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260925 02:43:17.675735 17267 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790304178325258-13942-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790304178325258-13942-0/minicluster-data/ts-1/wal
uuid: "2f22299df6604012a7aafb427f1f8ba8"
format_stamp: "Formatted at 2026-09-25 02:43:17 on dist-test-slave-kznw"
server_key: "bcc4b5941d75fb7e0109cf8fc72a6dab"
server_key_iv: "03dac388429eabfee895eed906d45b40"
server_key_version: "encryptionkey@0"
I20260925 02:43:17.675819 17267 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790304178325258-13942-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790304178325258-13942-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790304178325258-13942-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:43:17.689801 17267 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:43:17.692652 17267 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:43:17.692801 17267 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:43:17.693318 17267 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:43:17.694087 17267 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:43:17.694131 17267 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:17.694169 17267 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:43:17.694183 17267 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:17.702419 17267 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.130:43941
I20260925 02:43:17.702471 17397 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.130:43941 every 8 connection(s)
I20260925 02:43:17.703320 17267 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790304178325258-13942-0/minicluster-data/ts-1/data/info.pb
Sep 25 02:43:17 dist-test-slave-kznw krb5kdc[17023](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304197, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.130@KRBTEST.COM for kudu/127.13.157.190@KRBTEST.COM
I20260925 02:43:17.712536 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 17267
I20260925 02:43:17.712683 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790304178325258-13942-0/minicluster-data/ts-1/wal/instance
I20260925 02:43:17.712947 13942 external_mini_cluster.cc:1428] Setting key 96ee9fbe375fd1542b23e5a5ed004781
I20260925 02:43:17.716399 17109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:17.705096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.130:34697 (local address 127.13.157.190:42067)
0925 02:43:17.705269 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:17.705274 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:17.706047 (+ 773us) server_negotiation.cc:407] Connection header received
0925 02:43:17.707429 (+ 1382us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:17.707432 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:17.707481 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:17.707553 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:17.708705 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:17.709240 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:17.710345 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:17.710548 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:17.710621 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:17.713147 (+ 2526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:17.713150 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:17.713152 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:43:17.713179 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:17.714454 (+ 1275us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:17.714954 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:17.714958 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:17.714960 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:17.715036 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:17.715345 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:17.715348 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:17.715350 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:17.715526 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:17.715612 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:17.716115 (+ 503us) server_negotiation.cc:299] Negotiation successful
0925 02:43:17.716239 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260925 02:43:17.717214 17400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:17.705453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:42067 (local address 127.13.157.130:34697)
0925 02:43:17.705894 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:17.705938 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:17.707235 (+ 1297us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:17.707700 (+ 465us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:17.707707 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:17.708046 (+ 339us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:17.708561 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:17.708576 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:17.709358 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:17.709361 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:17.710253 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:17.710262 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:17.710666 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:17.710669 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:17.710811 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:17.711443 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:17.711461 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:17.712970 (+ 1509us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:43:17.714562 (+ 1592us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:17.714568 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:17.714583 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:17.714840 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:17.715117 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:17.715120 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:17.715122 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:17.715259 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:17.715631 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:43:17.715637 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:43:17.715876 (+ 239us) client_negotiation.cc:769] Sending connection context
0925 02:43:17.716043 (+ 167us) client_negotiation.cc:240] Negotiation successful
0925 02:43:17.716365 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":122,"threads_started":1}
I20260925 02:43:17.718416 17398 heartbeater.cc:347] Connected to a master server at 127.13.157.190:42067
I20260925 02:43:17.718643 17398 heartbeater.cc:464] Registering TS with master...
I20260925 02:43:17.719189 17398 heartbeater.cc:511] Master 127.13.157.190:42067 requested a full tablet report, sending...
I20260925 02:43:17.720283 17069 ts_manager.cc:194] Registered new tserver with Master: 2f22299df6604012a7aafb427f1f8ba8 (127.13.157.130:43941)
I20260925 02:43:17.721061 17069 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.157.130@KRBTEST.COM'} at 127.13.157.130:34697
WARNING: no policy specified for kudu/127.13.157.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.157.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.157.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.131.keytab.
Entry for principal kudu/127.13.157.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.131.keytab.
I20260925 02:43:17.738206 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790304178325258-13942-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790304178325258-13942-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790304178325258-13942-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790304178325258-13942-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.157.131:0
--local_ip_for_outbound_sockets=127.13.157.131
--webserver_interface=127.13.157.131
--webserver_port=0
--tserver_master_addrs=127.13.157.190:42067
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.131.keytab
--principal=kudu/127.13.157.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:38289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:43:17.847134 17403 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:43:17.847396 17403 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:43:17.847438 17403 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:43:17.851511 17403 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:43:17.851567 17403 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:43:17.851637 17403 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.157.131
I20260925 02:43:17.856858 17403 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:38289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790304178325258-13942-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790304178325258-13942-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.131.keytab
--principal=kudu/127.13.157.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.157.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790304178325258-13942-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.157.131
--webserver_port=0
--tserver_master_addrs=127.13.157.190:42067
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17403
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.157.131
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790304178325258-13942-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:43:17.857933 17403 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:43:17.858973 17403 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:43:17.868409 17408 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:17.868409 17411 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:17.868431 17409 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:43:17.868781 17403 server_base.cc:1034] running on GCE node
I20260925 02:43:17.869195 17403 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:43:17.869799 17403 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:43:17.870970 17403 hybrid_clock.cc:648] HybridClock initialized: now 1790304197870935 us; error 48 us; skew 500 ppm
Sep 25 02:43:17 dist-test-slave-kznw krb5kdc[17023](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304197, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:43:17.874205 17403 init.cc:377] Logged in from keytab as kudu/127.13.157.131@KRBTEST.COM (short username kudu)
I20260925 02:43:17.875411 17403 webserver.cc:460] Webserver started at http://127.13.157.131:45963/ using document root <none> and password file <none>
I20260925 02:43:17.876152 17403 fs_manager.cc:359] Metadata directory not provided
I20260925 02:43:17.876245 17403 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:43:17.876500 17403 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:43:17.879266 17403 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790304178325258-13942-0/minicluster-data/ts-2/data/instance:
uuid: "ad8a7bed567745a5a73b067b8ac422af"
format_stamp: "Formatted at 2026-09-25 02:43:17 on dist-test-slave-kznw"
server_key: "4b2c35cfd7203c57a8a372f12241663f"
server_key_iv: "74ebf8b2b67ebd98864a4fb5ccb6b32a"
server_key_version: "encryptionkey@0"
I20260925 02:43:17.880014 17403 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790304178325258-13942-0/minicluster-data/ts-2/wal/instance:
uuid: "ad8a7bed567745a5a73b067b8ac422af"
format_stamp: "Formatted at 2026-09-25 02:43:17 on dist-test-slave-kznw"
server_key: "4b2c35cfd7203c57a8a372f12241663f"
server_key_iv: "74ebf8b2b67ebd98864a4fb5ccb6b32a"
server_key_version: "encryptionkey@0"
I20260925 02:43:17.883970 17403 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260925 02:43:17.886411 17418 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:17.887795 17403 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260925 02:43:17.887919 17403 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790304178325258-13942-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790304178325258-13942-0/minicluster-data/ts-2/wal
uuid: "ad8a7bed567745a5a73b067b8ac422af"
format_stamp: "Formatted at 2026-09-25 02:43:17 on dist-test-slave-kznw"
server_key: "4b2c35cfd7203c57a8a372f12241663f"
server_key_iv: "74ebf8b2b67ebd98864a4fb5ccb6b32a"
server_key_version: "encryptionkey@0"
I20260925 02:43:17.888017 17403 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790304178325258-13942-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790304178325258-13942-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790304178325258-13942-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:43:17.908279 17403 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:43:17.911250 17403 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:43:17.911420 17403 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:43:17.912000 17403 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:43:17.912813 17403 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:43:17.912860 17403 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:17.912914 17403 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:43:17.912940 17403 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:17.921454 17403 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.131:36903
I20260925 02:43:17.921499 17533 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.131:36903 every 8 connection(s)
I20260925 02:43:17.922303 17403 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790304178325258-13942-0/minicluster-data/ts-2/data/info.pb
I20260925 02:43:17.924013 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 17403
I20260925 02:43:17.924198 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790304178325258-13942-0/minicluster-data/ts-2/wal/instance
I20260925 02:43:17.924492 13942 external_mini_cluster.cc:1428] Setting key 61061fe5fd0a167d828958db086b4c15
Sep 25 02:43:17 dist-test-slave-kznw krb5kdc[17023](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304197, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.131@KRBTEST.COM for kudu/127.13.157.190@KRBTEST.COM
I20260925 02:43:17.935221 17109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:17.924072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.131:36403 (local address 127.13.157.190:42067)
0925 02:43:17.924220 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:17.924224 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:17.925115 (+ 891us) server_negotiation.cc:407] Connection header received
0925 02:43:17.926729 (+ 1614us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:17.926732 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:17.926783 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:17.926847 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:17.927954 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:17.928510 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:17.929590 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:17.929799 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:17.929876 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:17.932201 (+ 2325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:17.932203 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:17.932206 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:43:17.932231 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:17.933363 (+ 1132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:17.933795 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:17.933798 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:17.933799 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:17.933869 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:17.934172 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:17.934175 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:17.934177 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:17.934365 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:17.934442 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:17.934929 (+ 487us) server_negotiation.cc:299] Negotiation successful
0925 02:43:17.935037 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260925 02:43:17.935957 17536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:17.924484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:42067 (local address 127.13.157.131:36403)
0925 02:43:17.924946 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:17.925001 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:17.926496 (+ 1495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:17.927023 (+ 527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:17.927030 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:17.927369 (+ 339us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:17.927842 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:17.927855 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:17.928622 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:17.928624 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:17.929496 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:17.929505 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:17.929971 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:17.929975 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:17.930126 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:17.930629 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:17.930644 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:17.932043 (+ 1399us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:43:17.933458 (+ 1415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:17.933462 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:17.933473 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:17.933723 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:17.933928 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:17.933930 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:17.933932 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:17.934053 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:17.934443 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:43:17.934448 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:43:17.934665 (+ 217us) client_negotiation.cc:769] Sending connection context
0925 02:43:17.934839 (+ 174us) client_negotiation.cc:240] Negotiation successful
0925 02:43:17.935120 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":119,"threads_started":1}
I20260925 02:43:17.937057 17534 heartbeater.cc:347] Connected to a master server at 127.13.157.190:42067
I20260925 02:43:17.937286 17534 heartbeater.cc:464] Registering TS with master...
I20260925 02:43:17.937811 17534 heartbeater.cc:511] Master 127.13.157.190:42067 requested a full tablet report, sending...
I20260925 02:43:17.938856 17069 ts_manager.cc:194] Registered new tserver with Master: ad8a7bed567745a5a73b067b8ac422af (127.13.157.131:36903)
I20260925 02:43:17.939399 17069 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.157.131@KRBTEST.COM'} at 127.13.157.131:36403
I20260925 02:43:17.950346 13942 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 25 02:43:17 dist-test-slave-kznw krb5kdc[17023](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304197, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Sep 25 02:43:17 dist-test-slave-kznw krb5kdc[17023](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304197, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.13.157.190@KRBTEST.COM
I20260925 02:43:17.974874 17109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:17.966848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59130 (local address 127.13.157.190:42067)
0925 02:43:17.967036 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:17.967041 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:17.967162 (+ 121us) server_negotiation.cc:407] Connection header received
0925 02:43:17.967296 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:17.967298 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:17.967346 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:17.967407 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:17.968173 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:17.968694 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:17.969542 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:17.969770 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:17.969849 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:17.972159 (+ 2310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:17.972162 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:17.972164 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:43:17.972191 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:17.973433 (+ 1242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:17.973802 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:17.973804 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:17.973805 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:17.973875 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:17.974091 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:17.974093 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:17.974094 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:17.974318 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:17.974421 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:17.974616 (+ 195us) server_negotiation.cc:299] Negotiation successful
0925 02:43:17.974712 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
W20260925 02:43:17.975317 17069 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:59130
I20260925 02:43:17.976096 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 17132
I20260925 02:43:17.981649 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 17267
I20260925 02:43:17.986980 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 17403
I20260925 02:43:17.991310 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 17036
2026-09-25T02:43:17Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2035 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790304178325258-13942-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 02:43:18 dist-test-slave-kznw krb5kdc[17546](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 02:43:18 dist-test-slave-kznw krb5kdc[17546](info): set up 2 sockets
Sep 25 02:43:18 dist-test-slave-kznw krb5kdc[17546](info): commencing operation
krb5kdc: starting...
W20260925 02:43:18.954493 13942 mini_kdc.cc:121] Time spent starting KDC: real 0.942s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790304178325258-13942-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790304178325258-13942-0/krb5kdc/test-user.keytab.
Sep 25 02:43:19 dist-test-slave-kznw krb5kdc[17546](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304199, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T02:43:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T02:43:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.157.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.157.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.157.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab.
Entry for principal kudu/127.13.157.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab.
I20260925 02:43:19.067992 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790304178325258-13942-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790304178325258-13942-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790304178325258-13942-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.157.190:45777
--webserver_interface=127.13.157.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab
--principal=kudu/127.13.157.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:39829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.157.190:45777
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:43:19.177119 17559 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:43:19.177395 17559 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:43:19.177435 17559 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:43:19.181433 17559 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:43:19.181494 17559 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:43:19.181514 17559 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:43:19.181528 17559 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:43:19.181542 17559 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:43:19.186878 17559 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:39829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790304178325258-13942-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790304178325258-13942-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.157.190:45777
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab
--principal=kudu/127.13.157.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.157.190:45777
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.157.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17559
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790304178325258-13942-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:43:19.187924 17559 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:43:19.188938 17559 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:43:19.197597 17564 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:19.197638 17565 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:19.197659 17567 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:43:19.197678 17559 server_base.cc:1034] running on GCE node
I20260925 02:43:19.198348 17559 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:43:19.199519 17559 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:43:19.200683 17559 hybrid_clock.cc:648] HybridClock initialized: now 1790304199200666 us; error 27 us; skew 500 ppm
Sep 25 02:43:19 dist-test-slave-kznw krb5kdc[17546](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304199, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:43:19.206509 17559 init.cc:377] Logged in from keytab as kudu/127.13.157.190@KRBTEST.COM (short username kudu)
I20260925 02:43:19.207569 17559 webserver.cc:460] Webserver started at http://127.13.157.190:38213/ using document root <none> and password file <none>
I20260925 02:43:19.208197 17559 fs_manager.cc:359] Metadata directory not provided
I20260925 02:43:19.208274 17559 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:43:19.208478 17559 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:43:19.210587 17559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790304178325258-13942-0/minicluster-data/master-0/data/instance:
uuid: "67392a8483d74b85be4f490ccf291432"
format_stamp: "Formatted at 2026-09-25 02:43:19 on dist-test-slave-kznw"
server_key: "01904965fce9ecc805399ae43111c3af"
server_key_iv: "b8c5c026ce769f131d5efc843934c1ca"
server_key_version: "encryptionkey@0"
I20260925 02:43:19.211169 17559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790304178325258-13942-0/minicluster-data/master-0/wal/instance:
uuid: "67392a8483d74b85be4f490ccf291432"
format_stamp: "Formatted at 2026-09-25 02:43:19 on dist-test-slave-kznw"
server_key: "01904965fce9ecc805399ae43111c3af"
server_key_iv: "b8c5c026ce769f131d5efc843934c1ca"
server_key_version: "encryptionkey@0"
I20260925 02:43:19.215022 17559 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260925 02:43:19.217399 17574 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:19.218617 17559 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260925 02:43:19.218734 17559 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790304178325258-13942-0/minicluster-data/master-0/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790304178325258-13942-0/minicluster-data/master-0/wal
uuid: "67392a8483d74b85be4f490ccf291432"
format_stamp: "Formatted at 2026-09-25 02:43:19 on dist-test-slave-kznw"
server_key: "01904965fce9ecc805399ae43111c3af"
server_key_iv: "b8c5c026ce769f131d5efc843934c1ca"
server_key_version: "encryptionkey@0"
I20260925 02:43:19.218829 17559 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790304178325258-13942-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790304178325258-13942-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790304178325258-13942-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:43:19.264216 17559 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:43:19.267198 17559 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:43:19.267377 17559 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:43:19.275149 17559 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.190:45777
I20260925 02:43:19.275199 17628 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.190:45777 every 8 connection(s)
I20260925 02:43:19.276175 17559 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
I20260925 02:43:19.278990 17629 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:43:19.284324 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 17559
I20260925 02:43:19.284453 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790304178325258-13942-0/minicluster-data/master-0/wal/instance
I20260925 02:43:19.284746 13942 external_mini_cluster.cc:1428] Setting key 2bba634fd6c3c6e22f13b0ce1b3be985
I20260925 02:43:19.284826 17629 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 67392a8483d74b85be4f490ccf291432: Bootstrap starting.
I20260925 02:43:19.287447 17629 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 67392a8483d74b85be4f490ccf291432: Neither blocks nor log segments found. Creating new log.
I20260925 02:43:19.288355 17629 log.cc:824] T 00000000000000000000000000000000 P 67392a8483d74b85be4f490ccf291432: Log is configured to *not* fsync() on all Append() calls
I20260925 02:43:19.290637 17629 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 67392a8483d74b85be4f490ccf291432: No bootstrap required, opened a new log
Sep 25 02:43:19 dist-test-slave-kznw krb5kdc[17546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304199, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.157.190@KRBTEST.COM
I20260925 02:43:19.294454 17629 raft_consensus.cc:348] T 00000000000000000000000000000000 P 67392a8483d74b85be4f490ccf291432 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67392a8483d74b85be4f490ccf291432" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 45777 } }
I20260925 02:43:19.294684 17629 raft_consensus.cc:374] T 00000000000000000000000000000000 P 67392a8483d74b85be4f490ccf291432 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:43:19.294735 17629 raft_consensus.cc:729] T 00000000000000000000000000000000 P 67392a8483d74b85be4f490ccf291432 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67392a8483d74b85be4f490ccf291432, State: Initialized, Role: FOLLOWER
I20260925 02:43:19.295172 17629 consensus_queue.cc:260] T 00000000000000000000000000000000 P 67392a8483d74b85be4f490ccf291432 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67392a8483d74b85be4f490ccf291432" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 45777 } }
I20260925 02:43:19.295296 17629 raft_consensus.cc:388] T 00000000000000000000000000000000 P 67392a8483d74b85be4f490ccf291432 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 02:43:19.295383 17629 raft_consensus.cc:482] T 00000000000000000000000000000000 P 67392a8483d74b85be4f490ccf291432 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 02:43:19.295485 17629 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 67392a8483d74b85be4f490ccf291432 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:43:19.296324 17629 raft_consensus.cc:504] T 00000000000000000000000000000000 P 67392a8483d74b85be4f490ccf291432 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67392a8483d74b85be4f490ccf291432" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 45777 } }
I20260925 02:43:19.296644 17629 leader_election.cc:302] T 00000000000000000000000000000000 P 67392a8483d74b85be4f490ccf291432 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 67392a8483d74b85be4f490ccf291432; no voters:
I20260925 02:43:19.296990 17629 leader_election.cc:288] T 00000000000000000000000000000000 P 67392a8483d74b85be4f490ccf291432 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 02:43:19.297076 17634 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 67392a8483d74b85be4f490ccf291432 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:43:19.297307 17634 raft_consensus.cc:686] T 00000000000000000000000000000000 P 67392a8483d74b85be4f490ccf291432 [term 1 LEADER]: Becoming Leader. State: Replica: 67392a8483d74b85be4f490ccf291432, State: Running, Role: LEADER
I20260925 02:43:19.297680 17634 consensus_queue.cc:237] T 00000000000000000000000000000000 P 67392a8483d74b85be4f490ccf291432 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67392a8483d74b85be4f490ccf291432" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 45777 } }
I20260925 02:43:19.298262 17629 sys_catalog.cc:564] T 00000000000000000000000000000000 P 67392a8483d74b85be4f490ccf291432 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:43:19.299340 17635 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67392a8483d74b85be4f490ccf291432 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "67392a8483d74b85be4f490ccf291432" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67392a8483d74b85be4f490ccf291432" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 45777 } } }
I20260925 02:43:19.299468 17635 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67392a8483d74b85be4f490ccf291432 [sys.catalog]: This master's current role is: LEADER
I20260925 02:43:19.299851 17636 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67392a8483d74b85be4f490ccf291432 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 67392a8483d74b85be4f490ccf291432. Latest consensus state: current_term: 1 leader_uuid: "67392a8483d74b85be4f490ccf291432" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67392a8483d74b85be4f490ccf291432" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 45777 } } }
I20260925 02:43:19.299968 17636 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67392a8483d74b85be4f490ccf291432 [sys.catalog]: This master's current role is: LEADER
I20260925 02:43:19.300107 17632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:19.286201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47708 (local address 127.13.157.190:45777)
0925 02:43:19.286631 (+ 430us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:19.286650 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:19.286694 (+ 44us) server_negotiation.cc:407] Connection header received
0925 02:43:19.287550 (+ 856us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:19.287574 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:19.288034 (+ 460us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:19.288425 (+ 391us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:19.289266 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:19.290713 (+ 1447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:19.291596 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:19.292299 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:19.292458 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:19.294854 (+ 2396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:19.294859 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:19.294872 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:43:19.294899 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:19.296602 (+ 1703us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:19.297151 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:19.297157 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:19.297163 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:19.297313 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:19.297602 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:19.297606 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:19.297607 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:19.298132 (+ 525us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:19.298334 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:19.298651 (+ 317us) server_negotiation.cc:299] Negotiation successful
0925 02:43:19.299002 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":280,"thread_start_us":120,"threads_started":1}
W20260925 02:43:19.303617 17650 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 67392a8483d74b85be4f490ccf291432: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 02:43:19.303702 17650 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 02:43:19.303758 17644 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:43:19.304606 17644 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:43:19.310335 17644 catalog_manager.cc:1347] Generated new cluster ID: 02bec9c678554e7caa7cec63a4ae3c76
I20260925 02:43:19.310395 17644 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 02:43:19.328403 17644 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 02:43:19.329649 17644 catalog_manager.cc:1504] Loading token signing keys...
I20260925 02:43:19.337528 17644 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 67392a8483d74b85be4f490ccf291432: Generated new TSK 0
I20260925 02:43:19.338369 17644 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.157.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.157.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.157.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.129.keytab.
Entry for principal kudu/127.13.157.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.129.keytab.
I20260925 02:43:19.370532 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790304178325258-13942-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790304178325258-13942-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790304178325258-13942-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790304178325258-13942-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.157.129:0
--local_ip_for_outbound_sockets=127.13.157.129
--webserver_interface=127.13.157.129
--webserver_port=0
--tserver_master_addrs=127.13.157.190:45777
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.129.keytab
--principal=kudu/127.13.157.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:39829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:43:19.478363 17656 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:43:19.478616 17656 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:43:19.478657 17656 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:43:19.482720 17656 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:43:19.482798 17656 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:43:19.482856 17656 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.157.129
I20260925 02:43:19.488309 17656 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:39829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790304178325258-13942-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790304178325258-13942-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.129.keytab
--principal=kudu/127.13.157.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.157.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790304178325258-13942-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.157.129
--webserver_port=0
--tserver_master_addrs=127.13.157.190:45777
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17656
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.157.129
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790304178325258-13942-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:43:19.489379 17656 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:43:19.490439 17656 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:43:19.500149 17664 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:19.500178 17661 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:43:19.500222 17656 server_base.cc:1034] running on GCE node
W20260925 02:43:19.500242 17662 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:43:19.500767 17656 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:43:19.501452 17656 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:43:19.502604 17656 hybrid_clock.cc:648] HybridClock initialized: now 1790304199502585 us; error 49 us; skew 500 ppm
Sep 25 02:43:19 dist-test-slave-kznw krb5kdc[17546](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304199, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:43:19.505331 17656 init.cc:377] Logged in from keytab as kudu/127.13.157.129@KRBTEST.COM (short username kudu)
I20260925 02:43:19.506167 17656 webserver.cc:460] Webserver started at http://127.13.157.129:42525/ using document root <none> and password file <none>
I20260925 02:43:19.506721 17656 fs_manager.cc:359] Metadata directory not provided
I20260925 02:43:19.506788 17656 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:43:19.507010 17656 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:43:19.509009 17656 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790304178325258-13942-0/minicluster-data/ts-0/data/instance:
uuid: "3a5ac256ca5f4f2f99aac64cfac6a14d"
format_stamp: "Formatted at 2026-09-25 02:43:19 on dist-test-slave-kznw"
server_key: "b6bbe1e01640ffe739de118f0543bf15"
server_key_iv: "49cbe3969941d381299b91029db95ffa"
server_key_version: "encryptionkey@0"
I20260925 02:43:19.509543 17656 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790304178325258-13942-0/minicluster-data/ts-0/wal/instance:
uuid: "3a5ac256ca5f4f2f99aac64cfac6a14d"
format_stamp: "Formatted at 2026-09-25 02:43:19 on dist-test-slave-kznw"
server_key: "b6bbe1e01640ffe739de118f0543bf15"
server_key_iv: "49cbe3969941d381299b91029db95ffa"
server_key_version: "encryptionkey@0"
I20260925 02:43:19.513154 17656 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260925 02:43:19.515631 17671 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:19.516974 17656 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260925 02:43:19.517148 17656 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790304178325258-13942-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790304178325258-13942-0/minicluster-data/ts-0/wal
uuid: "3a5ac256ca5f4f2f99aac64cfac6a14d"
format_stamp: "Formatted at 2026-09-25 02:43:19 on dist-test-slave-kznw"
server_key: "b6bbe1e01640ffe739de118f0543bf15"
server_key_iv: "49cbe3969941d381299b91029db95ffa"
server_key_version: "encryptionkey@0"
I20260925 02:43:19.517247 17656 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790304178325258-13942-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790304178325258-13942-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790304178325258-13942-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:43:19.534770 17656 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:43:19.537843 17656 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:43:19.538017 17656 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:43:19.538610 17656 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:43:19.539433 17656 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:43:19.539489 17656 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:19.539549 17656 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:43:19.539575 17656 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:19.547808 17656 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.129:44015
I20260925 02:43:19.547843 17785 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.129:44015 every 8 connection(s)
I20260925 02:43:19.548678 17656 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790304178325258-13942-0/minicluster-data/ts-0/data/info.pb
I20260925 02:43:19.556356 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 17656
I20260925 02:43:19.556476 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790304178325258-13942-0/minicluster-data/ts-0/wal/instance
I20260925 02:43:19.556778 13942 external_mini_cluster.cc:1428] Setting key 9c91cbca3c6ad5cd13f43ba52f69953f
Sep 25 02:43:19 dist-test-slave-kznw krb5kdc[17546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304199, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.129@KRBTEST.COM for kudu/127.13.157.190@KRBTEST.COM
I20260925 02:43:19.562315 17632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:19.550524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.129:42857 (local address 127.13.157.190:45777)
0925 02:43:19.550696 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:19.550699 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:19.551524 (+ 825us) server_negotiation.cc:407] Connection header received
0925 02:43:19.552927 (+ 1403us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:19.552931 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:19.552991 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:19.553066 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:19.554225 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:19.554762 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:19.555927 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:19.556163 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:19.556253 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:19.558940 (+ 2687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:19.558944 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:19.558947 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:43:19.558975 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:19.560317 (+ 1342us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:19.560887 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:19.560890 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:19.560891 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:19.560966 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:19.561307 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:19.561309 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:19.561310 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:19.561477 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:19.561554 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:19.562103 (+ 549us) server_negotiation.cc:299] Negotiation successful
0925 02:43:19.562195 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260925 02:43:19.563495 17788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:19.550879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:45777 (local address 127.13.157.129:42857)
0925 02:43:19.551376 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:19.551420 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:19.552701 (+ 1281us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:19.553255 (+ 554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:19.553263 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:19.553606 (+ 343us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:19.554069 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:19.554083 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:19.554881 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:19.554884 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:19.555800 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:19.555811 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:19.556303 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:19.556307 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:19.556475 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:19.557090 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:19.557106 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:19.558740 (+ 1634us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:43:19.560430 (+ 1690us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:19.560436 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:19.560452 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:19.560795 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:19.561049 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:19.561053 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:19.561055 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:19.561222 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:19.561580 (+ 358us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:43:19.561586 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:43:19.561885 (+ 299us) client_negotiation.cc:769] Sending connection context
0925 02:43:19.562084 (+ 199us) client_negotiation.cc:240] Negotiation successful
0925 02:43:19.562414 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":125,"threads_started":1}
I20260925 02:43:19.564839 17786 heartbeater.cc:347] Connected to a master server at 127.13.157.190:45777
I20260925 02:43:19.565104 17786 heartbeater.cc:464] Registering TS with master...
I20260925 02:43:19.565699 17786 heartbeater.cc:511] Master 127.13.157.190:45777 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.13.157.130@KRBTEST.COM; defaulting to no policy
I20260925 02:43:19.567323 17592 ts_manager.cc:194] Registered new tserver with Master: 3a5ac256ca5f4f2f99aac64cfac6a14d (127.13.157.129:44015)
I20260925 02:43:19.568531 17592 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.157.129@KRBTEST.COM'} at 127.13.157.129:42857
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.157.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.157.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.130.keytab.
Entry for principal kudu/127.13.157.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.130.keytab.
I20260925 02:43:19.584470 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790304178325258-13942-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790304178325258-13942-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790304178325258-13942-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790304178325258-13942-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.157.130:0
--local_ip_for_outbound_sockets=127.13.157.130
--webserver_interface=127.13.157.130
--webserver_port=0
--tserver_master_addrs=127.13.157.190:45777
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.130.keytab
--principal=kudu/127.13.157.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:39829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:43:19.693353 17792 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:43:19.693642 17792 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:43:19.693706 17792 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:43:19.697956 17792 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:43:19.698033 17792 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:43:19.698159 17792 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.157.130
I20260925 02:43:19.704160 17792 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:39829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790304178325258-13942-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790304178325258-13942-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.130.keytab
--principal=kudu/127.13.157.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.157.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790304178325258-13942-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.157.130
--webserver_port=0
--tserver_master_addrs=127.13.157.190:45777
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17792
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.157.130
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790304178325258-13942-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:43:19.705294 17792 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:43:19.706403 17792 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:43:19.716831 17798 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:19.716861 17800 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:43:19.716903 17792 server_base.cc:1034] running on GCE node
W20260925 02:43:19.716861 17797 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:43:19.717597 17792 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:43:19.718276 17792 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:43:19.719470 17792 hybrid_clock.cc:648] HybridClock initialized: now 1790304199719444 us; error 34 us; skew 500 ppm
Sep 25 02:43:19 dist-test-slave-kznw krb5kdc[17546](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304199, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:43:19.722421 17792 init.cc:377] Logged in from keytab as kudu/127.13.157.130@KRBTEST.COM (short username kudu)
I20260925 02:43:19.723366 17792 webserver.cc:460] Webserver started at http://127.13.157.130:36581/ using document root <none> and password file <none>
I20260925 02:43:19.723917 17792 fs_manager.cc:359] Metadata directory not provided
I20260925 02:43:19.723984 17792 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:43:19.724198 17792 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:43:19.726310 17792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790304178325258-13942-0/minicluster-data/ts-1/data/instance:
uuid: "49c6167c9433442f8f2faba8b47b63e2"
format_stamp: "Formatted at 2026-09-25 02:43:19 on dist-test-slave-kznw"
server_key: "6f61b8c1da43890c0759ca30e6508b2b"
server_key_iv: "c59834fb14505fd77f6f80c32e8d1e6b"
server_key_version: "encryptionkey@0"
I20260925 02:43:19.726898 17792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790304178325258-13942-0/minicluster-data/ts-1/wal/instance:
uuid: "49c6167c9433442f8f2faba8b47b63e2"
format_stamp: "Formatted at 2026-09-25 02:43:19 on dist-test-slave-kznw"
server_key: "6f61b8c1da43890c0759ca30e6508b2b"
server_key_iv: "c59834fb14505fd77f6f80c32e8d1e6b"
server_key_version: "encryptionkey@0"
I20260925 02:43:19.731196 17792 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260925 02:43:19.733644 17808 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:19.734934 17792 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260925 02:43:19.735105 17792 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790304178325258-13942-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790304178325258-13942-0/minicluster-data/ts-1/wal
uuid: "49c6167c9433442f8f2faba8b47b63e2"
format_stamp: "Formatted at 2026-09-25 02:43:19 on dist-test-slave-kznw"
server_key: "6f61b8c1da43890c0759ca30e6508b2b"
server_key_iv: "c59834fb14505fd77f6f80c32e8d1e6b"
server_key_version: "encryptionkey@0"
I20260925 02:43:19.735203 17792 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790304178325258-13942-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790304178325258-13942-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790304178325258-13942-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:43:19.757202 17792 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:43:19.761350 17792 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:43:19.761552 17792 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:43:19.762166 17792 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:43:19.763043 17792 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:43:19.763090 17792 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:19.763144 17792 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:43:19.763175 17792 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:19.772320 17792 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.130:34477
I20260925 02:43:19.772395 17921 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.130:34477 every 8 connection(s)
I20260925 02:43:19.773319 17792 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790304178325258-13942-0/minicluster-data/ts-1/data/info.pb
I20260925 02:43:19.780627 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 17792
I20260925 02:43:19.780769 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790304178325258-13942-0/minicluster-data/ts-1/wal/instance
I20260925 02:43:19.781077 13942 external_mini_cluster.cc:1428] Setting key 454b92ebf069a3262d73e01acc7aa101
Sep 25 02:43:19 dist-test-slave-kznw krb5kdc[17546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304199, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.130@KRBTEST.COM for kudu/127.13.157.190@KRBTEST.COM
I20260925 02:43:19.788419 17632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:19.775335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.130:46965 (local address 127.13.157.190:45777)
0925 02:43:19.775480 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:19.775483 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:19.776462 (+ 979us) server_negotiation.cc:407] Connection header received
0925 02:43:19.777854 (+ 1392us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:19.777858 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:19.777919 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:19.777994 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:19.779278 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:19.779826 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:19.781302 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:19.781548 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:19.781657 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:19.784520 (+ 2863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:19.784524 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:19.784526 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:43:19.784551 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:19.785940 (+ 1389us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:19.786564 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:19.786567 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:19.786569 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:19.786658 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:19.787059 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:19.787062 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:19.787063 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:19.787263 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:19.787344 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:19.788178 (+ 834us) server_negotiation.cc:299] Negotiation successful
0925 02:43:19.788292 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260925 02:43:19.789654 17924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:19.775783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:45777 (local address 127.13.157.130:46965)
0925 02:43:19.776313 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:19.776355 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:19.777634 (+ 1279us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:19.778195 (+ 561us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:19.778202 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:19.778571 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:19.779128 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:19.779146 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:19.779961 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:19.779964 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:19.781179 (+ 1215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:19.781191 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:19.781713 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:19.781716 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:19.781881 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:19.782619 (+ 738us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:19.782640 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:19.784341 (+ 1701us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:43:19.786057 (+ 1716us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:19.786063 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:19.786083 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:19.786441 (+ 358us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:19.786745 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:19.786748 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:19.786750 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:19.786957 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:19.787366 (+ 409us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:43:19.787373 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:43:19.787798 (+ 425us) client_negotiation.cc:769] Sending connection context
0925 02:43:19.788041 (+ 243us) client_negotiation.cc:240] Negotiation successful
0925 02:43:19.788425 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":126,"threads_started":1}
I20260925 02:43:19.790972 17922 heartbeater.cc:347] Connected to a master server at 127.13.157.190:45777
I20260925 02:43:19.791308 17922 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.13.157.131@KRBTEST.COM; defaulting to no policy
I20260925 02:43:19.791960 17922 heartbeater.cc:511] Master 127.13.157.190:45777 requested a full tablet report, sending...
I20260925 02:43:19.793195 17592 ts_manager.cc:194] Registered new tserver with Master: 49c6167c9433442f8f2faba8b47b63e2 (127.13.157.130:34477)
I20260925 02:43:19.794076 17592 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.157.130@KRBTEST.COM'} at 127.13.157.130:46965
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.157.131@KRBTEST.COM" created.
I20260925 02:43:20.570860 17786 heartbeater.cc:503] Master 127.13.157.190:45777 was elected leader, sending a full tablet report...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.157.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.131.keytab.
Entry for principal kudu/127.13.157.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.131.keytab.
W20260925 02:43:20.702993 13942 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.13.157.131: real 0.920s user 0.003s sys 0.000s
I20260925 02:43:20.703655 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790304178325258-13942-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790304178325258-13942-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790304178325258-13942-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790304178325258-13942-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.157.131:0
--local_ip_for_outbound_sockets=127.13.157.131
--webserver_interface=127.13.157.131
--webserver_port=0
--tserver_master_addrs=127.13.157.190:45777
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.131.keytab
--principal=kudu/127.13.157.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:39829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 02:43:20.796523 17922 heartbeater.cc:503] Master 127.13.157.190:45777 was elected leader, sending a full tablet report...
W20260925 02:43:20.819479 17927 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:43:20.819787 17927 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:43:20.819833 17927 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:43:20.824108 17927 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:43:20.824180 17927 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:43:20.824242 17927 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.157.131
I20260925 02:43:20.829722 17927 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:39829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790304178325258-13942-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790304178325258-13942-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.131.keytab
--principal=kudu/127.13.157.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.157.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790304178325258-13942-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.157.131
--webserver_port=0
--tserver_master_addrs=127.13.157.190:45777
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17927
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.157.131
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790304178325258-13942-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:43:20.830883 17927 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:43:20.832022 17927 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:43:20.842103 17935 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:20.842180 17933 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:20.842103 17932 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:43:20.842366 17927 server_base.cc:1034] running on GCE node
I20260925 02:43:20.842862 17927 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:43:20.843648 17927 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:43:20.844822 17927 hybrid_clock.cc:648] HybridClock initialized: now 1790304200844799 us; error 36 us; skew 500 ppm
Sep 25 02:43:20 dist-test-slave-kznw krb5kdc[17546](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304200, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:43:20.847698 17927 init.cc:377] Logged in from keytab as kudu/127.13.157.131@KRBTEST.COM (short username kudu)
I20260925 02:43:20.848675 17927 webserver.cc:460] Webserver started at http://127.13.157.131:40145/ using document root <none> and password file <none>
I20260925 02:43:20.849283 17927 fs_manager.cc:359] Metadata directory not provided
I20260925 02:43:20.849354 17927 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:43:20.849586 17927 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:43:20.851747 17927 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790304178325258-13942-0/minicluster-data/ts-2/data/instance:
uuid: "8991da475fc94d6487e78082ecc4e55c"
format_stamp: "Formatted at 2026-09-25 02:43:20 on dist-test-slave-kznw"
server_key: "bcc5da5e87b290b465905a16e9b5630d"
server_key_iv: "71d593c2ef909eace4ae367042caff8f"
server_key_version: "encryptionkey@0"
I20260925 02:43:20.852334 17927 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790304178325258-13942-0/minicluster-data/ts-2/wal/instance:
uuid: "8991da475fc94d6487e78082ecc4e55c"
format_stamp: "Formatted at 2026-09-25 02:43:20 on dist-test-slave-kznw"
server_key: "bcc5da5e87b290b465905a16e9b5630d"
server_key_iv: "71d593c2ef909eace4ae367042caff8f"
server_key_version: "encryptionkey@0"
I20260925 02:43:20.856349 17927 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260925 02:43:20.859433 17942 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:20.860961 17927 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260925 02:43:20.861071 17927 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790304178325258-13942-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790304178325258-13942-0/minicluster-data/ts-2/wal
uuid: "8991da475fc94d6487e78082ecc4e55c"
format_stamp: "Formatted at 2026-09-25 02:43:20 on dist-test-slave-kznw"
server_key: "bcc5da5e87b290b465905a16e9b5630d"
server_key_iv: "71d593c2ef909eace4ae367042caff8f"
server_key_version: "encryptionkey@0"
I20260925 02:43:20.861168 17927 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790304178325258-13942-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790304178325258-13942-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790304178325258-13942-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:43:20.898362 17927 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:43:20.901751 17927 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:43:20.901911 17927 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:43:20.902495 17927 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:43:20.903375 17927 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:43:20.903424 17927 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:20.903465 17927 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:43:20.903478 17927 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:20.912046 17927 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.131:38063
I20260925 02:43:20.912077 18056 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.131:38063 every 8 connection(s)
I20260925 02:43:20.912935 17927 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790304178325258-13942-0/minicluster-data/ts-2/data/info.pb
I20260925 02:43:20.920812 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 17927
I20260925 02:43:20.920950 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790304178325258-13942-0/minicluster-data/ts-2/wal/instance
I20260925 02:43:20.921242 13942 external_mini_cluster.cc:1428] Setting key 96eff074ad98ba9e4fba703cc39f4927
Sep 25 02:43:20 dist-test-slave-kznw krb5kdc[17546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304200, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.131@KRBTEST.COM for kudu/127.13.157.190@KRBTEST.COM
I20260925 02:43:20.927371 18059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:20.914785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.131:39579 (local address 127.13.157.190:45777)
0925 02:43:20.915086 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:20.915091 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:20.915948 (+ 857us) server_negotiation.cc:407] Connection header received
0925 02:43:20.917306 (+ 1358us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:20.917308 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:20.917359 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:20.917441 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:20.918710 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:20.919358 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:20.920655 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:20.920874 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:20.920980 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:20.923879 (+ 2899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:20.923882 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:20.923885 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:43:20.923915 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:20.925359 (+ 1444us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:20.925858 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:20.925860 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:20.925862 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:20.925939 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:20.926244 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:20.926246 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:20.926247 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:20.926431 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:20.926507 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:20.927051 (+ 544us) server_negotiation.cc:299] Negotiation successful
0925 02:43:20.927187 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":207,"thread_start_us":79,"threads_started":1}
I20260925 02:43:20.928248 18060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:20.915324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:45777 (local address 127.13.157.131:39579)
0925 02:43:20.915799 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:20.915842 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:20.917080 (+ 1238us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:20.917592 (+ 512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:20.917620 (+ 28us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:20.918026 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:20.918573 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:20.918587 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:20.919472 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:20.919475 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:20.920524 (+ 1049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:20.920535 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:20.921157 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:20.921162 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:20.921350 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:20.921925 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:20.921941 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:20.923735 (+ 1794us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:43:20.925484 (+ 1749us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:20.925489 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:20.925503 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:20.925777 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:20.926024 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:20.926026 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:20.926027 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:20.926175 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:20.926505 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:43:20.926510 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:43:20.926763 (+ 253us) client_negotiation.cc:769] Sending connection context
0925 02:43:20.926978 (+ 215us) client_negotiation.cc:240] Negotiation successful
0925 02:43:20.927319 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":118,"threads_started":1}
I20260925 02:43:20.929466 18057 heartbeater.cc:347] Connected to a master server at 127.13.157.190:45777
I20260925 02:43:20.929716 18057 heartbeater.cc:464] Registering TS with master...
I20260925 02:43:20.930238 18057 heartbeater.cc:511] Master 127.13.157.190:45777 requested a full tablet report, sending...
I20260925 02:43:20.931286 17592 ts_manager.cc:194] Registered new tserver with Master: 8991da475fc94d6487e78082ecc4e55c (127.13.157.131:38063)
I20260925 02:43:20.931865 17592 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.157.131@KRBTEST.COM'} at 127.13.157.131:39579
I20260925 02:43:20.935387 13942 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 25 02:43:20 dist-test-slave-kznw krb5kdc[17546](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304200, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 25 02:43:20 dist-test-slave-kznw krb5kdc[17546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304200, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.157.129@KRBTEST.COM
I20260925 02:43:20.966810 18065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:20.957078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36036 (local address 127.13.157.129:44015)
0925 02:43:20.957333 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:20.957342 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:20.957368 (+ 26us) server_negotiation.cc:407] Connection header received
0925 02:43:20.957421 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:20.957426 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:20.957634 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:20.957757 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:20.958601 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:20.959457 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:20.960430 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:20.960839 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:20.960938 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:20.963343 (+ 2405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:20.963353 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:20.963368 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:43:20.963396 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:20.964980 (+ 1584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:20.965431 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:20.965436 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:20.965438 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:20.965522 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:20.965809 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:20.965811 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:20.965812 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:20.966058 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:20.966293 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:20.966480 (+ 187us) server_negotiation.cc:299] Negotiation successful
0925 02:43:20.966612 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":60,"threads_started":1}
Sep 25 02:43:20 dist-test-slave-kznw krb5kdc[17546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304200, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.157.190@KRBTEST.COM
I20260925 02:43:20.978346 18059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:20.969720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47722 (local address 127.13.157.190:45777)
0925 02:43:20.969874 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:20.969878 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:20.969895 (+ 17us) server_negotiation.cc:407] Connection header received
0925 02:43:20.969994 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:20.969996 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:20.970037 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:20.970100 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:20.970696 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:20.971290 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:20.972086 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:20.972316 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:20.972394 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:20.975308 (+ 2914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:20.975311 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:20.975313 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:43:20.975341 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:20.976702 (+ 1361us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:20.977086 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:20.977089 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:20.977090 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:20.977165 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:20.977448 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:20.977450 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:20.977451 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:20.977699 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:20.977802 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:20.978077 (+ 275us) server_negotiation.cc:299] Negotiation successful
0925 02:43:20.978193 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
W20260925 02:43:20.978827 17592 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47722
I20260925 02:43:20.979532 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 17656
I20260925 02:43:20.985281 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 17792
I20260925 02:43:20.990795 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 17927
I20260925 02:43:20.996244 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 17559
2026-09-25T02:43:20Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (3005 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790304178325258-13942-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 02:43:21 dist-test-slave-kznw krb5kdc[18070](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 02:43:21 dist-test-slave-kznw krb5kdc[18070](info): set up 2 sockets
Sep 25 02:43:21 dist-test-slave-kznw krb5kdc[18070](info): commencing operation
krb5kdc: starting...
W20260925 02:43:22.518584 13942 mini_kdc.cc:121] Time spent starting KDC: real 1.501s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790304178325258-13942-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790304178325258-13942-0/krb5kdc/test-user.keytab.
Sep 25 02:43:22 dist-test-slave-kznw krb5kdc[18070](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304202, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T02:43:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T02:43:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.157.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.157.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.157.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab.
Entry for principal kudu/127.13.157.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab.
I20260925 02:43:22.636015 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790304178325258-13942-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790304178325258-13942-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790304178325258-13942-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.157.190:35859
--webserver_interface=127.13.157.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab
--principal=kudu/127.13.157.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:40203
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.157.190:35859
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:43:22.744501 18084 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:43:22.744764 18084 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:43:22.744827 18084 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:43:22.748837 18084 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:43:22.748898 18084 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:43:22.748925 18084 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:43:22.748948 18084 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:43:22.748977 18084 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:43:22.754253 18084 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:40203
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790304178325258-13942-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790304178325258-13942-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.157.190:35859
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab
--principal=kudu/127.13.157.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.157.190:35859
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.157.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18084
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790304178325258-13942-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:43:22.755391 18084 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:43:22.756405 18084 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:43:22.764763 18090 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:22.764801 18089 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:22.764775 18092 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:43:22.765018 18084 server_base.cc:1034] running on GCE node
I20260925 02:43:22.765645 18084 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:43:22.766808 18084 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:43:22.767956 18084 hybrid_clock.cc:648] HybridClock initialized: now 1790304202767933 us; error 36 us; skew 500 ppm
Sep 25 02:43:22 dist-test-slave-kznw krb5kdc[18070](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304202, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:43:22.770648 18084 init.cc:377] Logged in from keytab as kudu/127.13.157.190@KRBTEST.COM (short username kudu)
I20260925 02:43:22.771445 18084 fs_manager.cc:359] Metadata directory not provided
I20260925 02:43:22.771507 18084 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:43:22.771683 18084 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:43:22.773741 18084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790304178325258-13942-0/minicluster-data/master-0/data/instance:
uuid: "eb098a7380014c6881ca21db232e80c7"
format_stamp: "Formatted at 2026-09-25 02:43:22 on dist-test-slave-kznw"
server_key: "6943c3ee70098aeee6013ef55cefc27e"
server_key_iv: "174bdf1e1ece29d38dd3a932ee8e232b"
server_key_version: "encryptionkey@0"
I20260925 02:43:22.774268 18084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790304178325258-13942-0/minicluster-data/master-0/wal/instance:
uuid: "eb098a7380014c6881ca21db232e80c7"
format_stamp: "Formatted at 2026-09-25 02:43:22 on dist-test-slave-kznw"
server_key: "6943c3ee70098aeee6013ef55cefc27e"
server_key_iv: "174bdf1e1ece29d38dd3a932ee8e232b"
server_key_version: "encryptionkey@0"
I20260925 02:43:22.777915 18084 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260925 02:43:22.780467 18098 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:22.781781 18084 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260925 02:43:22.781905 18084 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790304178325258-13942-0/minicluster-data/master-0/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790304178325258-13942-0/minicluster-data/master-0/wal
uuid: "eb098a7380014c6881ca21db232e80c7"
format_stamp: "Formatted at 2026-09-25 02:43:22 on dist-test-slave-kznw"
server_key: "6943c3ee70098aeee6013ef55cefc27e"
server_key_iv: "174bdf1e1ece29d38dd3a932ee8e232b"
server_key_version: "encryptionkey@0"
I20260925 02:43:22.782004 18084 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790304178325258-13942-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790304178325258-13942-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790304178325258-13942-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:43:22.810279 18084 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:43:22.813526 18084 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:43:22.813725 18084 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:43:22.820663 18084 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.190:35859
I20260925 02:43:22.820924 18150 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.190:35859 every 8 connection(s)
I20260925 02:43:22.821329 18084 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
I20260925 02:43:22.822355 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 18084
I20260925 02:43:22.822474 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790304178325258-13942-0/minicluster-data/master-0/wal/instance
I20260925 02:43:22.822728 13942 external_mini_cluster.cc:1428] Setting key 4369e9c45a23a0c4cc2b14df76c5e854
I20260925 02:43:22.824797 18151 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:43:22.830482 18151 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb098a7380014c6881ca21db232e80c7: Bootstrap starting.
Sep 25 02:43:22 dist-test-slave-kznw krb5kdc[18070](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304202, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.157.190@KRBTEST.COM
I20260925 02:43:22.832875 18151 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eb098a7380014c6881ca21db232e80c7: Neither blocks nor log segments found. Creating new log.
I20260925 02:43:22.833587 18151 log.cc:824] T 00000000000000000000000000000000 P eb098a7380014c6881ca21db232e80c7: Log is configured to *not* fsync() on all Append() calls
I20260925 02:43:22.835563 18151 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb098a7380014c6881ca21db232e80c7: No bootstrap required, opened a new log
I20260925 02:43:22.836274 18154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:22.824155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44800 (local address 127.13.157.190:35859)
0925 02:43:22.824643 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:22.824664 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:22.824702 (+ 38us) server_negotiation.cc:407] Connection header received
0925 02:43:22.825389 (+ 687us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:22.825408 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:22.825775 (+ 367us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:22.826096 (+ 321us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:22.826957 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:22.827963 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:22.828818 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:22.829286 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:22.829434 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:22.831879 (+ 2445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:22.831884 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:22.831897 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:43:22.831926 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:22.833511 (+ 1585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:22.833960 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:22.833965 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:22.833969 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:22.834080 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:22.834335 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:22.834338 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:22.834339 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:22.834738 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:22.834934 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:22.835227 (+ 293us) server_negotiation.cc:299] Negotiation successful
0925 02:43:22.835506 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":331,"thread_start_us":154,"threads_started":1}
I20260925 02:43:22.839447 18151 raft_consensus.cc:348] T 00000000000000000000000000000000 P eb098a7380014c6881ca21db232e80c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb098a7380014c6881ca21db232e80c7" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 35859 } }
I20260925 02:43:22.839689 18151 raft_consensus.cc:374] T 00000000000000000000000000000000 P eb098a7380014c6881ca21db232e80c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:43:22.839785 18151 raft_consensus.cc:729] T 00000000000000000000000000000000 P eb098a7380014c6881ca21db232e80c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb098a7380014c6881ca21db232e80c7, State: Initialized, Role: FOLLOWER
I20260925 02:43:22.840198 18151 consensus_queue.cc:260] T 00000000000000000000000000000000 P eb098a7380014c6881ca21db232e80c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb098a7380014c6881ca21db232e80c7" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 35859 } }
I20260925 02:43:22.840340 18151 raft_consensus.cc:388] T 00000000000000000000000000000000 P eb098a7380014c6881ca21db232e80c7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 02:43:22.840404 18151 raft_consensus.cc:482] T 00000000000000000000000000000000 P eb098a7380014c6881ca21db232e80c7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 02:43:22.840499 18151 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eb098a7380014c6881ca21db232e80c7 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:43:22.841523 18151 raft_consensus.cc:504] T 00000000000000000000000000000000 P eb098a7380014c6881ca21db232e80c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb098a7380014c6881ca21db232e80c7" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 35859 } }
I20260925 02:43:22.841884 18151 leader_election.cc:302] T 00000000000000000000000000000000 P eb098a7380014c6881ca21db232e80c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eb098a7380014c6881ca21db232e80c7; no voters:
I20260925 02:43:22.842177 18151 leader_election.cc:288] T 00000000000000000000000000000000 P eb098a7380014c6881ca21db232e80c7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 02:43:22.842322 18156 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eb098a7380014c6881ca21db232e80c7 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:43:22.842530 18156 raft_consensus.cc:686] T 00000000000000000000000000000000 P eb098a7380014c6881ca21db232e80c7 [term 1 LEADER]: Becoming Leader. State: Replica: eb098a7380014c6881ca21db232e80c7, State: Running, Role: LEADER
I20260925 02:43:22.842917 18156 consensus_queue.cc:237] T 00000000000000000000000000000000 P eb098a7380014c6881ca21db232e80c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb098a7380014c6881ca21db232e80c7" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 35859 } }
I20260925 02:43:22.843415 18151 sys_catalog.cc:564] T 00000000000000000000000000000000 P eb098a7380014c6881ca21db232e80c7 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:43:22.844641 18157 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb098a7380014c6881ca21db232e80c7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eb098a7380014c6881ca21db232e80c7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb098a7380014c6881ca21db232e80c7" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 35859 } } }
I20260925 02:43:22.844691 18158 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb098a7380014c6881ca21db232e80c7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader eb098a7380014c6881ca21db232e80c7. Latest consensus state: current_term: 1 leader_uuid: "eb098a7380014c6881ca21db232e80c7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb098a7380014c6881ca21db232e80c7" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 35859 } } }
I20260925 02:43:22.844779 18158 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb098a7380014c6881ca21db232e80c7 [sys.catalog]: This master's current role is: LEADER
I20260925 02:43:22.844779 18157 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb098a7380014c6881ca21db232e80c7 [sys.catalog]: This master's current role is: LEADER
W20260925 02:43:22.848423 18171 catalog_manager.cc:1558] T 00000000000000000000000000000000 P eb098a7380014c6881ca21db232e80c7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 02:43:22.848515 18171 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 02:43:22.848624 18166 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:43:22.849500 18166 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:43:22.855450 18166 catalog_manager.cc:1347] Generated new cluster ID: c0231c14643542298d7e1afc6d948a2f
I20260925 02:43:22.855528 18166 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 02:43:22.871986 18166 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 02:43:22.873302 18166 catalog_manager.cc:1504] Loading token signing keys...
I20260925 02:43:22.882594 18166 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eb098a7380014c6881ca21db232e80c7: Generated new TSK 0
I20260925 02:43:22.883698 18166 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.157.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.157.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.157.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.129.keytab.
Entry for principal kudu/127.13.157.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.129.keytab.
I20260925 02:43:22.913836 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790304178325258-13942-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790304178325258-13942-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790304178325258-13942-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790304178325258-13942-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.157.129:0
--local_ip_for_outbound_sockets=127.13.157.129
--webserver_interface=127.13.157.129
--webserver_port=0
--tserver_master_addrs=127.13.157.190:35859
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.129.keytab
--principal=kudu/127.13.157.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:40203
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:43:23.024629 18177 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:43:23.024916 18177 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:43:23.024982 18177 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:43:23.029086 18177 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:43:23.029156 18177 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:43:23.029240 18177 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.157.129
I20260925 02:43:23.034614 18177 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:40203
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790304178325258-13942-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790304178325258-13942-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.129.keytab
--principal=kudu/127.13.157.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.157.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790304178325258-13942-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.157.129
--webserver_port=0
--tserver_master_addrs=127.13.157.190:35859
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18177
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.157.129
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790304178325258-13942-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:43:23.035709 18177 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:43:23.036727 18177 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:43:23.045987 18182 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:23.045990 18185 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:23.045987 18183 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:43:23.046190 18177 server_base.cc:1034] running on GCE node
I20260925 02:43:23.046677 18177 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:43:23.047340 18177 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:43:23.048513 18177 hybrid_clock.cc:648] HybridClock initialized: now 1790304203048495 us; error 33 us; skew 500 ppm
Sep 25 02:43:23 dist-test-slave-kznw krb5kdc[18070](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304203, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:43:23.051226 18177 init.cc:377] Logged in from keytab as kudu/127.13.157.129@KRBTEST.COM (short username kudu)
I20260925 02:43:23.051944 18177 fs_manager.cc:359] Metadata directory not provided
I20260925 02:43:23.052011 18177 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:43:23.052211 18177 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:43:23.054184 18177 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790304178325258-13942-0/minicluster-data/ts-0/data/instance:
uuid: "0a37dbe814c9478bb1679c092dcdb560"
format_stamp: "Formatted at 2026-09-25 02:43:23 on dist-test-slave-kznw"
server_key: "43d4067e6c35a64344110f0819070797"
server_key_iv: "0353c8fd827ced878a476f2537fa14a9"
server_key_version: "encryptionkey@0"
I20260925 02:43:23.054723 18177 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790304178325258-13942-0/minicluster-data/ts-0/wal/instance:
uuid: "0a37dbe814c9478bb1679c092dcdb560"
format_stamp: "Formatted at 2026-09-25 02:43:23 on dist-test-slave-kznw"
server_key: "43d4067e6c35a64344110f0819070797"
server_key_iv: "0353c8fd827ced878a476f2537fa14a9"
server_key_version: "encryptionkey@0"
I20260925 02:43:23.058272 18177 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260925 02:43:23.060829 18191 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:23.062129 18177 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260925 02:43:23.062259 18177 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790304178325258-13942-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790304178325258-13942-0/minicluster-data/ts-0/wal
uuid: "0a37dbe814c9478bb1679c092dcdb560"
format_stamp: "Formatted at 2026-09-25 02:43:23 on dist-test-slave-kznw"
server_key: "43d4067e6c35a64344110f0819070797"
server_key_iv: "0353c8fd827ced878a476f2537fa14a9"
server_key_version: "encryptionkey@0"
I20260925 02:43:23.062359 18177 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790304178325258-13942-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790304178325258-13942-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790304178325258-13942-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:43:23.086601 18177 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:43:23.089875 18177 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:43:23.090059 18177 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:43:23.090653 18177 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:43:23.091514 18177 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:43:23.091562 18177 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:23.091616 18177 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:43:23.091645 18177 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:23.100265 18177 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.129:36647
I20260925 02:43:23.100319 18305 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.129:36647 every 8 connection(s)
I20260925 02:43:23.100867 18177 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790304178325258-13942-0/minicluster-data/ts-0/data/info.pb
Sep 25 02:43:23 dist-test-slave-kznw krb5kdc[18070](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304203, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.129@KRBTEST.COM for kudu/127.13.157.190@KRBTEST.COM
I20260925 02:43:23.110649 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 18177
I20260925 02:43:23.110782 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790304178325258-13942-0/minicluster-data/ts-0/wal/instance
I20260925 02:43:23.111189 13942 external_mini_cluster.cc:1428] Setting key 69fe2c54461f8c696e3b2522332d2dbd
I20260925 02:43:23.114727 18154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:23.102561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.129:38697 (local address 127.13.157.190:35859)
0925 02:43:23.102731 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:23.102735 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:23.103622 (+ 887us) server_negotiation.cc:407] Connection header received
0925 02:43:23.104878 (+ 1256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:23.104880 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:23.104929 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:23.105004 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:23.106224 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:23.106783 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:23.108020 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:23.108268 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:23.108363 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:23.110978 (+ 2615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:23.110982 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:23.110984 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:43:23.111011 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:23.112444 (+ 1433us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:23.113028 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:23.113031 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:23.113032 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:23.113113 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:23.113501 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:23.113504 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:23.113506 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:23.113715 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:23.113801 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:23.114426 (+ 625us) server_negotiation.cc:299] Negotiation successful
0925 02:43:23.114559 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260925 02:43:23.115557 18308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:23.102999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:35859 (local address 127.13.157.129:38697)
0925 02:43:23.103464 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:23.103507 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:23.104669 (+ 1162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:23.105181 (+ 512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:23.105188 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:23.105544 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:23.106058 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:23.106071 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:23.106898 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:23.106923 (+ 25us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:23.107877 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:23.107888 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:23.108414 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:23.108417 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:23.108566 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:23.109126 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:23.109146 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:23.110770 (+ 1624us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:43:23.112568 (+ 1798us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:23.112573 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:23.112592 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:23.112934 (+ 342us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:23.113206 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:23.113209 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:23.113211 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:23.113387 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:23.113816 (+ 429us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:43:23.113823 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:43:23.114141 (+ 318us) client_negotiation.cc:769] Sending connection context
0925 02:43:23.114350 (+ 209us) client_negotiation.cc:240] Negotiation successful
0925 02:43:23.114659 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":107,"threads_started":1}
I20260925 02:43:23.116698 18306 heartbeater.cc:347] Connected to a master server at 127.13.157.190:35859
I20260925 02:43:23.116964 18306 heartbeater.cc:464] Registering TS with master...
I20260925 02:43:23.117516 18306 heartbeater.cc:511] Master 127.13.157.190:35859 requested a full tablet report, sending...
I20260925 02:43:23.119194 18115 ts_manager.cc:194] Registered new tserver with Master: 0a37dbe814c9478bb1679c092dcdb560 (127.13.157.129:36647)
WARNING: no policy specified for kudu/127.13.157.130@KRBTEST.COM; defaulting to no policy
I20260925 02:43:23.120314 18115 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.157.129@KRBTEST.COM'} at 127.13.157.129:38697
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.157.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.157.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.130.keytab.
Entry for principal kudu/127.13.157.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.130.keytab.
I20260925 02:43:23.136168 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790304178325258-13942-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790304178325258-13942-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790304178325258-13942-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790304178325258-13942-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.157.130:0
--local_ip_for_outbound_sockets=127.13.157.130
--webserver_interface=127.13.157.130
--webserver_port=0
--tserver_master_addrs=127.13.157.190:35859
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.130.keytab
--principal=kudu/127.13.157.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:40203
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:43:23.243804 18311 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:43:23.244045 18311 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:43:23.244097 18311 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:43:23.248100 18311 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:43:23.248163 18311 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:43:23.248245 18311 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.157.130
I20260925 02:43:23.253468 18311 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:40203
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790304178325258-13942-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790304178325258-13942-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.130.keytab
--principal=kudu/127.13.157.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.157.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790304178325258-13942-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.157.130
--webserver_port=0
--tserver_master_addrs=127.13.157.190:35859
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18311
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.157.130
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790304178325258-13942-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:43:23.254478 18311 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:43:23.255501 18311 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 02:43:23.264669 18311 server_base.cc:1034] running on GCE node
W20260925 02:43:23.264605 18317 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:23.264647 18320 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:23.264617 18318 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:43:23.265247 18311 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:43:23.265852 18311 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:43:23.266991 18311 hybrid_clock.cc:648] HybridClock initialized: now 1790304203266964 us; error 37 us; skew 500 ppm
Sep 25 02:43:23 dist-test-slave-kznw krb5kdc[18070](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304203, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:43:23.269584 18311 init.cc:377] Logged in from keytab as kudu/127.13.157.130@KRBTEST.COM (short username kudu)
I20260925 02:43:23.270321 18311 fs_manager.cc:359] Metadata directory not provided
I20260925 02:43:23.270388 18311 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:43:23.270577 18311 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:43:23.272608 18311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790304178325258-13942-0/minicluster-data/ts-1/data/instance:
uuid: "e84cd30de1064af49a3f309c96da9b5c"
format_stamp: "Formatted at 2026-09-25 02:43:23 on dist-test-slave-kznw"
server_key: "8fd0c93a6ceea34a50d206bf9b6717b9"
server_key_iv: "8b545ff0f12ca2e10e64f9bfae3749e7"
server_key_version: "encryptionkey@0"
I20260925 02:43:23.273139 18311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790304178325258-13942-0/minicluster-data/ts-1/wal/instance:
uuid: "e84cd30de1064af49a3f309c96da9b5c"
format_stamp: "Formatted at 2026-09-25 02:43:23 on dist-test-slave-kznw"
server_key: "8fd0c93a6ceea34a50d206bf9b6717b9"
server_key_iv: "8b545ff0f12ca2e10e64f9bfae3749e7"
server_key_version: "encryptionkey@0"
I20260925 02:43:23.276654 18311 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260925 02:43:23.279111 18326 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:23.280309 18311 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260925 02:43:23.280434 18311 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790304178325258-13942-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790304178325258-13942-0/minicluster-data/ts-1/wal
uuid: "e84cd30de1064af49a3f309c96da9b5c"
format_stamp: "Formatted at 2026-09-25 02:43:23 on dist-test-slave-kznw"
server_key: "8fd0c93a6ceea34a50d206bf9b6717b9"
server_key_iv: "8b545ff0f12ca2e10e64f9bfae3749e7"
server_key_version: "encryptionkey@0"
I20260925 02:43:23.280531 18311 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790304178325258-13942-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790304178325258-13942-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790304178325258-13942-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:43:23.305513 18311 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:43:23.308715 18311 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:43:23.308923 18311 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:43:23.309588 18311 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:43:23.310519 18311 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:43:23.310572 18311 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:23.310635 18311 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:43:23.310667 18311 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:23.320399 18311 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.130:40343
I20260925 02:43:23.320449 18441 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.130:40343 every 8 connection(s)
I20260925 02:43:23.321038 18311 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790304178325258-13942-0/minicluster-data/ts-1/data/info.pb
I20260925 02:43:23.322278 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 18311
I20260925 02:43:23.322402 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790304178325258-13942-0/minicluster-data/ts-1/wal/instance
I20260925 02:43:23.322736 13942 external_mini_cluster.cc:1428] Setting key a5fae31046c489607af82c95b14d3d93
Sep 25 02:43:23 dist-test-slave-kznw krb5kdc[18070](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304203, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.130@KRBTEST.COM for kudu/127.13.157.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.157.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.157.131@KRBTEST.COM" created.
I20260925 02:43:23.335814 18154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:23.322869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.130:51051 (local address 127.13.157.190:35859)
0925 02:43:23.323055 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:23.323060 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:23.323886 (+ 826us) server_negotiation.cc:407] Connection header received
0925 02:43:23.325553 (+ 1667us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:23.325556 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:23.325609 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:23.325674 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:23.327081 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:23.327833 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:23.329102 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:23.329456 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:23.329572 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:23.332142 (+ 2570us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:23.332146 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:23.332149 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:43:23.332181 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:23.333530 (+ 1349us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:23.334072 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:23.334076 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:23.334077 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:23.334210 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:23.334621 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:23.334623 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:23.334625 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:23.334811 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:23.334889 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:23.335525 (+ 636us) server_negotiation.cc:299] Negotiation successful
0925 02:43:23.335673 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260925 02:43:23.336884 18444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:23.323266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:35859 (local address 127.13.157.130:51051)
0925 02:43:23.323745 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:23.323789 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:23.325282 (+ 1493us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:23.325863 (+ 581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:23.325873 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:23.326370 (+ 497us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:23.326881 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:23.326896 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:23.327983 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:23.327986 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:23.328967 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:23.328977 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:23.329605 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:23.329607 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:23.329746 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:23.330360 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:23.330376 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:23.331948 (+ 1572us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:43:23.333651 (+ 1703us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:23.333656 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:23.333670 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:23.333944 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:23.334337 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:23.334341 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:23.334343 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:23.334514 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:23.334894 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:43:23.334899 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:43:23.335226 (+ 327us) client_negotiation.cc:769] Sending connection context
0925 02:43:23.335414 (+ 188us) client_negotiation.cc:240] Negotiation successful
0925 02:43:23.335796 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":120,"threads_started":1}
I20260925 02:43:23.338301 18442 heartbeater.cc:347] Connected to a master server at 127.13.157.190:35859
I20260925 02:43:23.338577 18442 heartbeater.cc:464] Registering TS with master...
I20260925 02:43:23.339159 18442 heartbeater.cc:511] Master 127.13.157.190:35859 requested a full tablet report, sending...
I20260925 02:43:23.340373 18115 ts_manager.cc:194] Registered new tserver with Master: e84cd30de1064af49a3f309c96da9b5c (127.13.157.130:40343)
I20260925 02:43:23.340947 18115 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.157.130@KRBTEST.COM'} at 127.13.157.130:51051
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.157.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.131.keytab.
Entry for principal kudu/127.13.157.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.131.keytab.
I20260925 02:43:23.351609 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790304178325258-13942-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790304178325258-13942-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790304178325258-13942-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790304178325258-13942-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.157.131:0
--local_ip_for_outbound_sockets=127.13.157.131
--webserver_interface=127.13.157.131
--webserver_port=0
--tserver_master_addrs=127.13.157.190:35859
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.131.keytab
--principal=kudu/127.13.157.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:40203
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:43:23.459883 18447 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:43:23.460148 18447 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:43:23.460207 18447 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:43:23.464375 18447 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:43:23.464430 18447 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:43:23.464509 18447 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.157.131
I20260925 02:43:23.469655 18447 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:40203
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790304178325258-13942-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790304178325258-13942-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.131.keytab
--principal=kudu/127.13.157.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.157.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790304178325258-13942-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.157.131
--webserver_port=0
--tserver_master_addrs=127.13.157.190:35859
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18447
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.157.131
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790304178325258-13942-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:43:23.470640 18447 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:43:23.471662 18447 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:43:23.480656 18455 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:23.480659 18453 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:23.480733 18452 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:43:23.480827 18447 server_base.cc:1034] running on GCE node
I20260925 02:43:23.481335 18447 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:43:23.481948 18447 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:43:23.483135 18447 hybrid_clock.cc:648] HybridClock initialized: now 1790304203483118 us; error 30 us; skew 500 ppm
Sep 25 02:43:23 dist-test-slave-kznw krb5kdc[18070](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304203, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:43:23.486027 18447 init.cc:377] Logged in from keytab as kudu/127.13.157.131@KRBTEST.COM (short username kudu)
I20260925 02:43:23.486826 18447 fs_manager.cc:359] Metadata directory not provided
I20260925 02:43:23.486891 18447 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:43:23.487126 18447 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:43:23.489362 18447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790304178325258-13942-0/minicluster-data/ts-2/data/instance:
uuid: "bc2cdbc65c474f7dafc4f491925aab6c"
format_stamp: "Formatted at 2026-09-25 02:43:23 on dist-test-slave-kznw"
server_key: "29d5159072e1c162d7c99c925b00e480"
server_key_iv: "ab2e54238378a5ce39c71cc40d185073"
server_key_version: "encryptionkey@0"
I20260925 02:43:23.489993 18447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790304178325258-13942-0/minicluster-data/ts-2/wal/instance:
uuid: "bc2cdbc65c474f7dafc4f491925aab6c"
format_stamp: "Formatted at 2026-09-25 02:43:23 on dist-test-slave-kznw"
server_key: "29d5159072e1c162d7c99c925b00e480"
server_key_iv: "ab2e54238378a5ce39c71cc40d185073"
server_key_version: "encryptionkey@0"
I20260925 02:43:23.493816 18447 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260925 02:43:23.496376 18461 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:23.497661 18447 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260925 02:43:23.497771 18447 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790304178325258-13942-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790304178325258-13942-0/minicluster-data/ts-2/wal
uuid: "bc2cdbc65c474f7dafc4f491925aab6c"
format_stamp: "Formatted at 2026-09-25 02:43:23 on dist-test-slave-kznw"
server_key: "29d5159072e1c162d7c99c925b00e480"
server_key_iv: "ab2e54238378a5ce39c71cc40d185073"
server_key_version: "encryptionkey@0"
I20260925 02:43:23.497865 18447 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790304178325258-13942-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790304178325258-13942-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790304178325258-13942-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:43:23.539712 18447 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:43:23.542649 18447 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:43:23.542824 18447 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:43:23.543443 18447 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:43:23.544296 18447 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:43:23.544343 18447 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:23.544394 18447 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:43:23.544417 18447 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:23.553339 18447 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.131:36751
I20260925 02:43:23.553367 18575 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.131:36751 every 8 connection(s)
I20260925 02:43:23.553941 18447 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790304178325258-13942-0/minicluster-data/ts-2/data/info.pb
I20260925 02:43:23.557749 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 18447
I20260925 02:43:23.557868 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790304178325258-13942-0/minicluster-data/ts-2/wal/instance
I20260925 02:43:23.558212 13942 external_mini_cluster.cc:1428] Setting key 03ff3fba58cbeb48fde3b6b8712aceaa
Sep 25 02:43:23 dist-test-slave-kznw krb5kdc[18070](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304203, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.131@KRBTEST.COM for kudu/127.13.157.190@KRBTEST.COM
I20260925 02:43:23.567564 18154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:23.555816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.131:34535 (local address 127.13.157.190:35859)
0925 02:43:23.555956 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:23.555959 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:23.556854 (+ 895us) server_negotiation.cc:407] Connection header received
0925 02:43:23.558456 (+ 1602us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:23.558459 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:23.558521 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:23.558603 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:23.560012 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:23.560648 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:23.561781 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:23.561995 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:23.562074 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:23.564404 (+ 2330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:23.564407 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:23.564409 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:43:23.564434 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:23.565657 (+ 1223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:23.566161 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:23.566164 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:23.566166 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:23.566250 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:23.566546 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:23.566548 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:23.566549 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:23.566735 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:23.566824 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:23.567334 (+ 510us) server_negotiation.cc:299] Negotiation successful
0925 02:43:23.567438 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260925 02:43:23.568440 18578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:23.556186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:35859 (local address 127.13.157.131:34535)
0925 02:43:23.556683 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:23.556729 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:23.558130 (+ 1401us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:23.558822 (+ 692us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:23.558830 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:23.559286 (+ 456us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:23.559844 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:23.559864 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:23.560771 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:23.560774 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:23.561678 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:23.561688 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:23.562128 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:23.562132 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:23.562275 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:23.562801 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:23.562818 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:23.564235 (+ 1417us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:43:23.565764 (+ 1529us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:23.565768 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:23.565781 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:23.566064 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:23.566334 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:23.566336 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:23.566337 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:23.566478 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:23.566824 (+ 346us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:43:23.566829 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:43:23.567111 (+ 282us) client_negotiation.cc:769] Sending connection context
0925 02:43:23.567290 (+ 179us) client_negotiation.cc:240] Negotiation successful
0925 02:43:23.567546 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":119,"threads_started":1}
I20260925 02:43:23.569681 18576 heartbeater.cc:347] Connected to a master server at 127.13.157.190:35859
I20260925 02:43:23.569926 18576 heartbeater.cc:464] Registering TS with master...
I20260925 02:43:23.570387 18576 heartbeater.cc:511] Master 127.13.157.190:35859 requested a full tablet report, sending...
I20260925 02:43:23.571415 18115 ts_manager.cc:194] Registered new tserver with Master: bc2cdbc65c474f7dafc4f491925aab6c (127.13.157.131:36751)
I20260925 02:43:23.572013 18115 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.157.131@KRBTEST.COM'} at 127.13.157.131:34535
I20260925 02:43:23.572664 13942 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 02:43:23.581657 18154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:23.574816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44808 (local address 127.13.157.190:35859)
0925 02:43:23.574976 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:23.574980 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:23.575132 (+ 152us) server_negotiation.cc:407] Connection header received
0925 02:43:23.575220 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:23.575222 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:23.575261 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:23.575321 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:23.575916 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:23.576415 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:23.577255 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:23.577464 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:23.577559 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:23.578770 (+ 1211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:23.578773 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:23.578775 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:43:23.578796 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:23.580155 (+ 1359us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:23.580539 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:23.580542 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:23.580543 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:23.580615 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:23.580899 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:23.580901 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:23.580902 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:23.581138 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:23.581236 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:23.581400 (+ 164us) server_negotiation.cc:299] Negotiation successful
0925 02:43:23.581495 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260925 02:43:23.584570 18114 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44808:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 02:43:23.587634 18114 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 02:43:23.602317 18591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:23.596884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50568 (local address 127.13.157.129:36647)
0925 02:43:23.597219 (+ 335us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:23.597226 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:23.597432 (+ 206us) server_negotiation.cc:407] Connection header received
0925 02:43:23.597637 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:23.597643 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:23.597815 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:23.597950 (+ 135us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:43:23.598216 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:23.599414 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:23.600882 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:23.601622 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:23.601837 (+ 215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:23.602002 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:23.602058 (+ 56us) server_negotiation.cc:299] Negotiation successful
0925 02:43:23.602155 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":250,"thread_start_us":71,"threads_started":1}
I20260925 02:43:23.602367 18589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:23.596273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.129:36647 (local address 127.0.0.1:50568)
0925 02:43:23.597336 (+ 1063us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:23.597346 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:23.597478 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:23.597951 (+ 473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:23.597957 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:23.597978 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:43:23.598115 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:23.598122 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:23.599690 (+ 1568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:23.599694 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:23.600771 (+ 1077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:23.600781 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:23.601737 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:23.601741 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:23.601847 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:23.601891 (+ 44us) client_negotiation.cc:769] Sending connection context
0925 02:43:23.601995 (+ 104us) client_negotiation.cc:240] Negotiation successful
0925 02:43:23.602114 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1013,"mutex_wait_us":29,"thread_start_us":86,"threads_started":1}
I20260925 02:43:23.602880 18590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:23.596065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.130:40343 (local address 127.0.0.1:42446)
0925 02:43:23.597125 (+ 1060us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:23.597173 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:23.597623 (+ 450us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:23.598182 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:23.598185 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:23.598206 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:43:23.598308 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:23.598314 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:23.599408 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:23.599412 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:23.601091 (+ 1679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:23.601102 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:23.602491 (+ 1389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:23.602495 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:23.602583 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:23.602605 (+ 22us) client_negotiation.cc:769] Sending connection context
0925 02:43:23.602677 (+ 72us) client_negotiation.cc:240] Negotiation successful
0925 02:43:23.602748 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":888,"mutex_wait_us":121,"thread_start_us":85,"threads_started":1}
I20260925 02:43:23.603147 18588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:23.596182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42446 (local address 127.13.157.130:40343)
0925 02:43:23.596496 (+ 314us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:23.596508 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:23.597259 (+ 751us) server_negotiation.cc:407] Connection header received
0925 02:43:23.597753 (+ 494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:23.597758 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:23.597908 (+ 150us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:23.598023 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:43:23.598428 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:23.599292 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:23.601216 (+ 1924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:23.602066 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:23.602180 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:23.602292 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:23.602863 (+ 571us) server_negotiation.cc:299] Negotiation successful
0925 02:43:23.602999 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":207,"thread_start_us":97,"threads_started":1}
I20260925 02:43:23.603899 18592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:23.596789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43952 (local address 127.13.157.131:36751)
0925 02:43:23.597224 (+ 435us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:23.597234 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:23.597482 (+ 248us) server_negotiation.cc:407] Connection header received
0925 02:43:23.597618 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:23.597623 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:23.597824 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:23.597970 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:43:23.598379 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:23.599522 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:23.601274 (+ 1752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:23.601940 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:23.602043 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:23.602175 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:23.603649 (+ 1474us) server_negotiation.cc:299] Negotiation successful
0925 02:43:23.603758 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":163,"threads_started":1}
I20260925 02:43:23.604238 18587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:23.596619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.131:36751 (local address 127.0.0.1:43952)
0925 02:43:23.597395 (+ 776us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:23.597408 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:23.597516 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:23.598125 (+ 609us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:23.598128 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:23.598146 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:43:23.598264 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:23.598271 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:23.599622 (+ 1351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:23.599626 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:23.601091 (+ 1465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:23.601102 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:23.602239 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:23.602242 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:23.602336 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:23.602358 (+ 22us) client_negotiation.cc:769] Sending connection context
0925 02:43:23.604035 (+ 1677us) client_negotiation.cc:240] Negotiation successful
0925 02:43:23.604116 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":714,"mutex_wait_us":138,"thread_start_us":98,"threads_started":1}
I20260925 02:43:23.605720 18239 tablet_service.cc:1467] Processing CreateTablet for tablet ba2ca6ed195744bc89c5f074c976baf8 (DEFAULT_TABLE table=test-table [id=488d445fe7014af99303aa457aefae55]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:43:23.605772 18375 tablet_service.cc:1467] Processing CreateTablet for tablet ba2ca6ed195744bc89c5f074c976baf8 (DEFAULT_TABLE table=test-table [id=488d445fe7014af99303aa457aefae55]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:43:23.606655 18239 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba2ca6ed195744bc89c5f074c976baf8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:43:23.606842 18375 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba2ca6ed195744bc89c5f074c976baf8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:43:23.606954 18510 tablet_service.cc:1467] Processing CreateTablet for tablet ba2ca6ed195744bc89c5f074c976baf8 (DEFAULT_TABLE table=test-table [id=488d445fe7014af99303aa457aefae55]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:43:23.608031 18510 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba2ca6ed195744bc89c5f074c976baf8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:43:23.612922 18593 tablet_bootstrap.cc:492] T ba2ca6ed195744bc89c5f074c976baf8 P 0a37dbe814c9478bb1679c092dcdb560: Bootstrap starting.
I20260925 02:43:23.614465 18594 tablet_bootstrap.cc:492] T ba2ca6ed195744bc89c5f074c976baf8 P e84cd30de1064af49a3f309c96da9b5c: Bootstrap starting.
I20260925 02:43:23.615232 18595 tablet_bootstrap.cc:492] T ba2ca6ed195744bc89c5f074c976baf8 P bc2cdbc65c474f7dafc4f491925aab6c: Bootstrap starting.
I20260925 02:43:23.615410 18593 tablet_bootstrap.cc:654] T ba2ca6ed195744bc89c5f074c976baf8 P 0a37dbe814c9478bb1679c092dcdb560: Neither blocks nor log segments found. Creating new log.
I20260925 02:43:23.616490 18593 log.cc:824] T ba2ca6ed195744bc89c5f074c976baf8 P 0a37dbe814c9478bb1679c092dcdb560: Log is configured to *not* fsync() on all Append() calls
I20260925 02:43:23.616667 18594 tablet_bootstrap.cc:654] T ba2ca6ed195744bc89c5f074c976baf8 P e84cd30de1064af49a3f309c96da9b5c: Neither blocks nor log segments found. Creating new log.
I20260925 02:43:23.617237 18595 tablet_bootstrap.cc:654] T ba2ca6ed195744bc89c5f074c976baf8 P bc2cdbc65c474f7dafc4f491925aab6c: Neither blocks nor log segments found. Creating new log.
I20260925 02:43:23.617885 18594 log.cc:824] T ba2ca6ed195744bc89c5f074c976baf8 P e84cd30de1064af49a3f309c96da9b5c: Log is configured to *not* fsync() on all Append() calls
I20260925 02:43:23.618511 18595 log.cc:824] T ba2ca6ed195744bc89c5f074c976baf8 P bc2cdbc65c474f7dafc4f491925aab6c: Log is configured to *not* fsync() on all Append() calls
I20260925 02:43:23.618860 18593 tablet_bootstrap.cc:492] T ba2ca6ed195744bc89c5f074c976baf8 P 0a37dbe814c9478bb1679c092dcdb560: No bootstrap required, opened a new log
I20260925 02:43:23.619087 18593 ts_tablet_manager.cc:1397] T ba2ca6ed195744bc89c5f074c976baf8 P 0a37dbe814c9478bb1679c092dcdb560: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.001s
I20260925 02:43:23.619756 18594 tablet_bootstrap.cc:492] T ba2ca6ed195744bc89c5f074c976baf8 P e84cd30de1064af49a3f309c96da9b5c: No bootstrap required, opened a new log
I20260925 02:43:23.619927 18594 ts_tablet_manager.cc:1397] T ba2ca6ed195744bc89c5f074c976baf8 P e84cd30de1064af49a3f309c96da9b5c: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260925 02:43:23.620357 18595 tablet_bootstrap.cc:492] T ba2ca6ed195744bc89c5f074c976baf8 P bc2cdbc65c474f7dafc4f491925aab6c: No bootstrap required, opened a new log
I20260925 02:43:23.620523 18595 ts_tablet_manager.cc:1397] T ba2ca6ed195744bc89c5f074c976baf8 P bc2cdbc65c474f7dafc4f491925aab6c: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260925 02:43:23.622761 18594 raft_consensus.cc:348] T ba2ca6ed195744bc89c5f074c976baf8 P e84cd30de1064af49a3f309c96da9b5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e84cd30de1064af49a3f309c96da9b5c" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 40343 } } peers { permanent_uuid: "0a37dbe814c9478bb1679c092dcdb560" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 36647 } } peers { permanent_uuid: "bc2cdbc65c474f7dafc4f491925aab6c" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 36751 } }
I20260925 02:43:23.623024 18594 raft_consensus.cc:374] T ba2ca6ed195744bc89c5f074c976baf8 P e84cd30de1064af49a3f309c96da9b5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:43:23.623095 18594 raft_consensus.cc:729] T ba2ca6ed195744bc89c5f074c976baf8 P e84cd30de1064af49a3f309c96da9b5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e84cd30de1064af49a3f309c96da9b5c, State: Initialized, Role: FOLLOWER
I20260925 02:43:23.622929 18593 raft_consensus.cc:348] T ba2ca6ed195744bc89c5f074c976baf8 P 0a37dbe814c9478bb1679c092dcdb560 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e84cd30de1064af49a3f309c96da9b5c" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 40343 } } peers { permanent_uuid: "0a37dbe814c9478bb1679c092dcdb560" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 36647 } } peers { permanent_uuid: "bc2cdbc65c474f7dafc4f491925aab6c" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 36751 } }
I20260925 02:43:23.623167 18593 raft_consensus.cc:374] T ba2ca6ed195744bc89c5f074c976baf8 P 0a37dbe814c9478bb1679c092dcdb560 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:43:23.623224 18593 raft_consensus.cc:729] T ba2ca6ed195744bc89c5f074c976baf8 P 0a37dbe814c9478bb1679c092dcdb560 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a37dbe814c9478bb1679c092dcdb560, State: Initialized, Role: FOLLOWER
I20260925 02:43:23.623558 18594 consensus_queue.cc:260] T ba2ca6ed195744bc89c5f074c976baf8 P e84cd30de1064af49a3f309c96da9b5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e84cd30de1064af49a3f309c96da9b5c" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 40343 } } peers { permanent_uuid: "0a37dbe814c9478bb1679c092dcdb560" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 36647 } } peers { permanent_uuid: "bc2cdbc65c474f7dafc4f491925aab6c" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 36751 } }
I20260925 02:43:23.623636 18593 consensus_queue.cc:260] T ba2ca6ed195744bc89c5f074c976baf8 P 0a37dbe814c9478bb1679c092dcdb560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e84cd30de1064af49a3f309c96da9b5c" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 40343 } } peers { permanent_uuid: "0a37dbe814c9478bb1679c092dcdb560" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 36647 } } peers { permanent_uuid: "bc2cdbc65c474f7dafc4f491925aab6c" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 36751 } }
I20260925 02:43:23.624176 18442 heartbeater.cc:503] Master 127.13.157.190:35859 was elected leader, sending a full tablet report...
I20260925 02:43:23.624112 18595 raft_consensus.cc:348] T ba2ca6ed195744bc89c5f074c976baf8 P bc2cdbc65c474f7dafc4f491925aab6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e84cd30de1064af49a3f309c96da9b5c" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 40343 } } peers { permanent_uuid: "0a37dbe814c9478bb1679c092dcdb560" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 36647 } } peers { permanent_uuid: "bc2cdbc65c474f7dafc4f491925aab6c" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 36751 } }
I20260925 02:43:23.624392 18595 raft_consensus.cc:374] T ba2ca6ed195744bc89c5f074c976baf8 P bc2cdbc65c474f7dafc4f491925aab6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:43:23.624441 18595 raft_consensus.cc:729] T ba2ca6ed195744bc89c5f074c976baf8 P bc2cdbc65c474f7dafc4f491925aab6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc2cdbc65c474f7dafc4f491925aab6c, State: Initialized, Role: FOLLOWER
I20260925 02:43:23.624539 18594 ts_tablet_manager.cc:1428] T ba2ca6ed195744bc89c5f074c976baf8 P e84cd30de1064af49a3f309c96da9b5c: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260925 02:43:23.624804 18595 consensus_queue.cc:260] T ba2ca6ed195744bc89c5f074c976baf8 P bc2cdbc65c474f7dafc4f491925aab6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e84cd30de1064af49a3f309c96da9b5c" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 40343 } } peers { permanent_uuid: "0a37dbe814c9478bb1679c092dcdb560" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 36647 } } peers { permanent_uuid: "bc2cdbc65c474f7dafc4f491925aab6c" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 36751 } }
I20260925 02:43:23.625285 18593 ts_tablet_manager.cc:1428] T ba2ca6ed195744bc89c5f074c976baf8 P 0a37dbe814c9478bb1679c092dcdb560: Time spent starting tablet: real 0.006s user 0.003s sys 0.002s
I20260925 02:43:23.625641 18306 heartbeater.cc:503] Master 127.13.157.190:35859 was elected leader, sending a full tablet report...
I20260925 02:43:23.625675 18595 ts_tablet_manager.cc:1428] T ba2ca6ed195744bc89c5f074c976baf8 P bc2cdbc65c474f7dafc4f491925aab6c: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260925 02:43:23.625978 18576 heartbeater.cc:503] Master 127.13.157.190:35859 was elected leader, sending a full tablet report...
I20260925 02:43:23.744858 18601 raft_consensus.cc:482] T ba2ca6ed195744bc89c5f074c976baf8 P bc2cdbc65c474f7dafc4f491925aab6c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 02:43:23.745097 18601 raft_consensus.cc:504] T ba2ca6ed195744bc89c5f074c976baf8 P bc2cdbc65c474f7dafc4f491925aab6c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e84cd30de1064af49a3f309c96da9b5c" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 40343 } } peers { permanent_uuid: "0a37dbe814c9478bb1679c092dcdb560" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 36647 } } peers { permanent_uuid: "bc2cdbc65c474f7dafc4f491925aab6c" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 36751 } }
I20260925 02:43:23.746431 18601 leader_election.cc:288] T ba2ca6ed195744bc89c5f074c976baf8 P bc2cdbc65c474f7dafc4f491925aab6c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e84cd30de1064af49a3f309c96da9b5c (127.13.157.130:40343), 0a37dbe814c9478bb1679c092dcdb560 (127.13.157.129:36647)
I20260925 02:43:23.751253 18588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:23.746755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.131:55941 (local address 127.13.157.130:40343)
0925 02:43:23.746862 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:23.746866 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:23.746880 (+ 14us) server_negotiation.cc:407] Connection header received
0925 02:43:23.747055 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:23.747058 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:23.747111 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:23.747187 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:43:23.747442 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:23.748217 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:23.749926 (+ 1709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:23.750558 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:23.750672 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:23.750724 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:23.751045 (+ 321us) server_negotiation.cc:299] Negotiation successful
0925 02:43:23.751122 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260925 02:43:23.751262 18591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:23.746796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.131:33777 (local address 127.13.157.129:36647)
0925 02:43:23.746936 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:23.746941 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:23.747185 (+ 244us) server_negotiation.cc:407] Connection header received
0925 02:43:23.747264 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:23.747267 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:23.747318 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:23.747378 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:43:23.747621 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:23.748337 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:23.749926 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:23.750545 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:23.750672 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:23.750724 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:23.751045 (+ 321us) server_negotiation.cc:299] Negotiation successful
0925 02:43:23.751161 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260925 02:43:23.751315 18578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:23.746639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.130:40343 (local address 127.13.157.131:55941)
0925 02:43:23.746809 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:23.746824 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:23.746933 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:23.747207 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:23.747210 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:23.747224 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:43:23.747338 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:23.747342 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:23.748347 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:23.748350 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:23.749783 (+ 1433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:23.749794 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:23.750708 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:23.750713 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:23.750863 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:23.750935 (+ 72us) client_negotiation.cc:769] Sending connection context
0925 02:43:23.751011 (+ 76us) client_negotiation.cc:240] Negotiation successful
0925 02:43:23.751165 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"mutex_wait_us":88}
I20260925 02:43:23.751324 18602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:23.746676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.129:36647 (local address 127.13.157.131:33777)
0925 02:43:23.747091 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:23.747104 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:23.747187 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:23.747408 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:23.747411 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:23.747427 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:43:23.747527 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:23.747531 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:23.748470 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:23.748474 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:23.749783 (+ 1309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:23.749794 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:23.750713 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:23.750716 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:23.750887 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:23.750950 (+ 63us) client_negotiation.cc:769] Sending connection context
0925 02:43:23.751011 (+ 61us) client_negotiation.cc:240] Negotiation successful
0925 02:43:23.751171 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":353,"mutex_wait_us":181,"thread_start_us":115,"threads_started":1}
I20260925 02:43:23.752005 18259 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba2ca6ed195744bc89c5f074c976baf8" candidate_uuid: "bc2cdbc65c474f7dafc4f491925aab6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a37dbe814c9478bb1679c092dcdb560" is_pre_election: true
I20260925 02:43:23.752081 18395 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba2ca6ed195744bc89c5f074c976baf8" candidate_uuid: "bc2cdbc65c474f7dafc4f491925aab6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e84cd30de1064af49a3f309c96da9b5c" is_pre_election: true
I20260925 02:43:23.752329 18259 raft_consensus.cc:2454] T ba2ca6ed195744bc89c5f074c976baf8 P 0a37dbe814c9478bb1679c092dcdb560 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc2cdbc65c474f7dafc4f491925aab6c in term 0.
I20260925 02:43:23.752389 18395 raft_consensus.cc:2454] T ba2ca6ed195744bc89c5f074c976baf8 P e84cd30de1064af49a3f309c96da9b5c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc2cdbc65c474f7dafc4f491925aab6c in term 0.
I20260925 02:43:23.752859 18463 leader_election.cc:302] T ba2ca6ed195744bc89c5f074c976baf8 P bc2cdbc65c474f7dafc4f491925aab6c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a37dbe814c9478bb1679c092dcdb560, bc2cdbc65c474f7dafc4f491925aab6c; no voters:
I20260925 02:43:23.753191 18601 raft_consensus.cc:2781] T ba2ca6ed195744bc89c5f074c976baf8 P bc2cdbc65c474f7dafc4f491925aab6c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 02:43:23.753273 18601 raft_consensus.cc:482] T ba2ca6ed195744bc89c5f074c976baf8 P bc2cdbc65c474f7dafc4f491925aab6c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 02:43:23.753302 18601 raft_consensus.cc:3037] T ba2ca6ed195744bc89c5f074c976baf8 P bc2cdbc65c474f7dafc4f491925aab6c [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:43:23.754366 18601 raft_consensus.cc:504] T ba2ca6ed195744bc89c5f074c976baf8 P bc2cdbc65c474f7dafc4f491925aab6c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e84cd30de1064af49a3f309c96da9b5c" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 40343 } } peers { permanent_uuid: "0a37dbe814c9478bb1679c092dcdb560" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 36647 } } peers { permanent_uuid: "bc2cdbc65c474f7dafc4f491925aab6c" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 36751 } }
I20260925 02:43:23.754709 18601 leader_election.cc:288] T ba2ca6ed195744bc89c5f074c976baf8 P bc2cdbc65c474f7dafc4f491925aab6c [CANDIDATE]: Term 1 election: Requested vote from peers e84cd30de1064af49a3f309c96da9b5c (127.13.157.130:40343), 0a37dbe814c9478bb1679c092dcdb560 (127.13.157.129:36647)
I20260925 02:43:23.755218 18259 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba2ca6ed195744bc89c5f074c976baf8" candidate_uuid: "bc2cdbc65c474f7dafc4f491925aab6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a37dbe814c9478bb1679c092dcdb560"
I20260925 02:43:23.755229 18395 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba2ca6ed195744bc89c5f074c976baf8" candidate_uuid: "bc2cdbc65c474f7dafc4f491925aab6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e84cd30de1064af49a3f309c96da9b5c"
I20260925 02:43:23.755333 18259 raft_consensus.cc:3037] T ba2ca6ed195744bc89c5f074c976baf8 P 0a37dbe814c9478bb1679c092dcdb560 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:43:23.755333 18395 raft_consensus.cc:3037] T ba2ca6ed195744bc89c5f074c976baf8 P e84cd30de1064af49a3f309c96da9b5c [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:43:23.756346 18259 raft_consensus.cc:2454] T ba2ca6ed195744bc89c5f074c976baf8 P 0a37dbe814c9478bb1679c092dcdb560 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc2cdbc65c474f7dafc4f491925aab6c in term 1.
I20260925 02:43:23.756361 18395 raft_consensus.cc:2454] T ba2ca6ed195744bc89c5f074c976baf8 P e84cd30de1064af49a3f309c96da9b5c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc2cdbc65c474f7dafc4f491925aab6c in term 1.
I20260925 02:43:23.756656 18465 leader_election.cc:302] T ba2ca6ed195744bc89c5f074c976baf8 P bc2cdbc65c474f7dafc4f491925aab6c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc2cdbc65c474f7dafc4f491925aab6c, e84cd30de1064af49a3f309c96da9b5c; no voters:
I20260925 02:43:23.756820 18601 raft_consensus.cc:2781] T ba2ca6ed195744bc89c5f074c976baf8 P bc2cdbc65c474f7dafc4f491925aab6c [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:43:23.757042 18601 raft_consensus.cc:686] T ba2ca6ed195744bc89c5f074c976baf8 P bc2cdbc65c474f7dafc4f491925aab6c [term 1 LEADER]: Becoming Leader. State: Replica: bc2cdbc65c474f7dafc4f491925aab6c, State: Running, Role: LEADER
I20260925 02:43:23.757362 18601 consensus_queue.cc:237] T ba2ca6ed195744bc89c5f074c976baf8 P bc2cdbc65c474f7dafc4f491925aab6c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e84cd30de1064af49a3f309c96da9b5c" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 40343 } } peers { permanent_uuid: "0a37dbe814c9478bb1679c092dcdb560" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 36647 } } peers { permanent_uuid: "bc2cdbc65c474f7dafc4f491925aab6c" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 36751 } }
I20260925 02:43:23.760610 18115 catalog_manager.cc:5495] T ba2ca6ed195744bc89c5f074c976baf8 P bc2cdbc65c474f7dafc4f491925aab6c reported cstate change: term changed from 0 to 1, leader changed from <none> to bc2cdbc65c474f7dafc4f491925aab6c (127.13.157.131). New cstate: current_term: 1 leader_uuid: "bc2cdbc65c474f7dafc4f491925aab6c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e84cd30de1064af49a3f309c96da9b5c" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 40343 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a37dbe814c9478bb1679c092dcdb560" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 36647 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc2cdbc65c474f7dafc4f491925aab6c" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 36751 } health_report { overall_health: HEALTHY } } }
I20260925 02:43:23.791662 18592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:23.787860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43954 (local address 127.13.157.131:36751)
0925 02:43:23.787972 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:23.787976 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:23.787991 (+ 15us) server_negotiation.cc:407] Connection header received
0925 02:43:23.788128 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:23.788130 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:23.788186 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:23.788280 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 02:43:23.788549 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:23.789084 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:23.790038 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:23.790323 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:23.790420 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:23.790767 (+ 347us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 02:43:23.791298 (+ 531us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 02:43:23.791402 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:23.791456 (+ 54us) server_negotiation.cc:299] Negotiation successful
0925 02:43:23.791512 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260925 02:43:23.797634 18395 raft_consensus.cc:1261] T ba2ca6ed195744bc89c5f074c976baf8 P e84cd30de1064af49a3f309c96da9b5c [term 1 FOLLOWER]: Refusing update from remote peer bc2cdbc65c474f7dafc4f491925aab6c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:43:23.797642 18259 raft_consensus.cc:1261] T ba2ca6ed195744bc89c5f074c976baf8 P 0a37dbe814c9478bb1679c092dcdb560 [term 1 FOLLOWER]: Refusing update from remote peer bc2cdbc65c474f7dafc4f491925aab6c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:43:23.798338 18603 consensus_queue.cc:1035] T ba2ca6ed195744bc89c5f074c976baf8 P bc2cdbc65c474f7dafc4f491925aab6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e84cd30de1064af49a3f309c96da9b5c" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 40343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:43:23.798529 18601 consensus_queue.cc:1035] T ba2ca6ed195744bc89c5f074c976baf8 P bc2cdbc65c474f7dafc4f491925aab6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a37dbe814c9478bb1679c092dcdb560" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 36647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260925 02:43:23.804796 18577 tablet.cc:2369] T ba2ca6ed195744bc89c5f074c976baf8 P bc2cdbc65c474f7dafc4f491925aab6c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 02:43:23.807891 18607 mvcc.cc:205] Tried to move back new op lower bound from 7333086018747871232 to 7333086018592739328. Current Snapshot: MvccSnapshot[applied={T|T < 7333086018747871232}]
I20260925 02:43:23.809944 18608 mvcc.cc:205] Tried to move back new op lower bound from 7333086018747871232 to 7333086018592739328. Current Snapshot: MvccSnapshot[applied={T|T < 7333086018747871232}]
I20260925 02:43:23.815341 18113 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44808:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 02:43:23.815495 18113 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44808:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 02:43:23.817823 18113 catalog_manager.cc:5782] T 00000000000000000000000000000000 P eb098a7380014c6881ca21db232e80c7: Sending DeleteTablet for 3 replicas of tablet ba2ca6ed195744bc89c5f074c976baf8
I20260925 02:43:23.818552 18239 tablet_service.cc:1514] Processing DeleteTablet for tablet ba2ca6ed195744bc89c5f074c976baf8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:43:23 UTC) from {username='kudu', principal='kudu/127.13.157.190@KRBTEST.COM'} at 127.0.0.1:50568
I20260925 02:43:23.818549 18510 tablet_service.cc:1514] Processing DeleteTablet for tablet ba2ca6ed195744bc89c5f074c976baf8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:43:23 UTC) from {username='kudu', principal='kudu/127.13.157.190@KRBTEST.COM'} at 127.0.0.1:43952
I20260925 02:43:23.818554 18375 tablet_service.cc:1514] Processing DeleteTablet for tablet ba2ca6ed195744bc89c5f074c976baf8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:43:23 UTC) from {username='kudu', principal='kudu/127.13.157.190@KRBTEST.COM'} at 127.0.0.1:42446
I20260925 02:43:23.819224 18616 tablet_replica.cc:330] T ba2ca6ed195744bc89c5f074c976baf8 P 0a37dbe814c9478bb1679c092dcdb560: stopping tablet replica
I20260925 02:43:23.819249 18615 tablet_replica.cc:330] T ba2ca6ed195744bc89c5f074c976baf8 P bc2cdbc65c474f7dafc4f491925aab6c: stopping tablet replica
I20260925 02:43:23.819559 18616 raft_consensus.cc:2229] T ba2ca6ed195744bc89c5f074c976baf8 P 0a37dbe814c9478bb1679c092dcdb560 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 02:43:23.819689 18615 raft_consensus.cc:2229] T ba2ca6ed195744bc89c5f074c976baf8 P bc2cdbc65c474f7dafc4f491925aab6c [term 1 LEADER]: Raft consensus shutting down.
I20260925 02:43:23.819903 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 18177
I20260925 02:43:23.820089 18615 raft_consensus.cc:2258] T ba2ca6ed195744bc89c5f074c976baf8 P bc2cdbc65c474f7dafc4f491925aab6c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:43:23.822602 18615 ts_tablet_manager.cc:1905] T ba2ca6ed195744bc89c5f074c976baf8 P bc2cdbc65c474f7dafc4f491925aab6c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:43:23.825388 18615 ts_tablet_manager.cc:1918] T ba2ca6ed195744bc89c5f074c976baf8 P bc2cdbc65c474f7dafc4f491925aab6c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 02:43:23.825678 18617 tablet_replica.cc:330] T ba2ca6ed195744bc89c5f074c976baf8 P e84cd30de1064af49a3f309c96da9b5c: stopping tablet replica
I20260925 02:43:23.825852 18615 log.cc:1196] T ba2ca6ed195744bc89c5f074c976baf8 P bc2cdbc65c474f7dafc4f491925aab6c: Deleting WAL directory at /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790304178325258-13942-0/minicluster-data/ts-2/wal/wals/ba2ca6ed195744bc89c5f074c976baf8
I20260925 02:43:23.826263 18615 ts_tablet_manager.cc:1939] T ba2ca6ed195744bc89c5f074c976baf8 P bc2cdbc65c474f7dafc4f491925aab6c: Deleting consensus metadata
I20260925 02:43:23.826640 18617 raft_consensus.cc:2229] T ba2ca6ed195744bc89c5f074c976baf8 P e84cd30de1064af49a3f309c96da9b5c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 02:43:23.827061 18617 raft_consensus.cc:2258] T ba2ca6ed195744bc89c5f074c976baf8 P e84cd30de1064af49a3f309c96da9b5c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:43:23.828373 18099 catalog_manager.cc:4841] TS bc2cdbc65c474f7dafc4f491925aab6c (127.13.157.131:36751): tablet ba2ca6ed195744bc89c5f074c976baf8 (table test-table [id=488d445fe7014af99303aa457aefae55]) successfully deleted
I20260925 02:43:23.829604 18617 ts_tablet_manager.cc:1905] T ba2ca6ed195744bc89c5f074c976baf8 P e84cd30de1064af49a3f309c96da9b5c: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260925 02:43:23.829939 18100 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.157.129:36647 (error 108)
I20260925 02:43:23.830690 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 18311
I20260925 02:43:23.830897 18587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:23.830638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.129:36647 (local address 127.0.0.1:50570)
0925 02:43:23.830756 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:23.830822 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.129:36647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260925 02:43:23.832432 18100 catalog_manager.cc:4584] TS 0a37dbe814c9478bb1679c092dcdb560 (127.13.157.129:36647): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ba2ca6ed195744bc89c5f074c976baf8: Network error: Client connection negotiation failed: client connection to 127.13.157.129:36647: connect: Connection refused (error 111)
I20260925 02:43:23.841074 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 18447
I20260925 02:43:23.842578 18587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:23.841396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.130:40343 (local address 127.0.0.1:42458)
0925 02:43:23.842405 (+ 1009us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:23.842479 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.130:40343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":914}
I20260925 02:43:23.846925 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 18084
2026-09-25T02:43:23Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2850 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-09-25T02:43:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T02:43:23Z Disabled control of system clock
I20260925 02:43:23.876143 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790304178325258-13942-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790304178325258-13942-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790304178325258-13942-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.157.190:45399
--webserver_interface=127.13.157.190
--webserver_port=0
--builtin_ntp_servers=127.13.157.148:38553
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.157.190:45399
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260925 02:43:23.985608 18622 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:43:23.985855 18622 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:43:23.985891 18622 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:43:23.989797 18622 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:43:23.989857 18622 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:43:23.989876 18622 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:43:23.989892 18622 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:43:23.989905 18622 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:43:23.994650 18622 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:38553
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790304178325258-13942-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790304178325258-13942-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.157.190:45399
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.157.190:45399
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.157.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18622
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790304178325258-13942-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:43:23.995731 18622 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:43:23.997062 18622 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:43:24.005780 18628 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:24.005791 18627 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:43:24.005846 18622 server_base.cc:1034] running on GCE node
W20260925 02:43:24.005791 18630 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:43:24.006594 18622 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:43:24.008085 18622 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:43:24.009284 18622 hybrid_clock.cc:648] HybridClock initialized: now 1790304204009256 us; error 39 us; skew 500 ppm
I20260925 02:43:24.011654 18622 webserver.cc:460] Webserver started at http://127.13.157.190:41207/ using document root <none> and password file <none>
I20260925 02:43:24.012338 18622 fs_manager.cc:359] Metadata directory not provided
I20260925 02:43:24.012427 18622 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:43:24.012697 18622 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:43:24.015082 18622 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790304178325258-13942-0/minicluster-data/master-0/data/instance:
uuid: "3d0d39a173b54b1eb89375b7670be95f"
format_stamp: "Formatted at 2026-09-25 02:43:24 on dist-test-slave-kznw"
server_key: "b4f9feb7abc147ba97f4507db4736d42"
server_key_iv: "4f93ea37b60ee683856bdd774653d2a1"
server_key_version: "encryptionkey@0"
I20260925 02:43:24.015722 18622 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790304178325258-13942-0/minicluster-data/master-0/wal/instance:
uuid: "3d0d39a173b54b1eb89375b7670be95f"
format_stamp: "Formatted at 2026-09-25 02:43:24 on dist-test-slave-kznw"
server_key: "b4f9feb7abc147ba97f4507db4736d42"
server_key_iv: "4f93ea37b60ee683856bdd774653d2a1"
server_key_version: "encryptionkey@0"
I20260925 02:43:24.019565 18622 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260925 02:43:24.022089 18636 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:24.023434 18622 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260925 02:43:24.023552 18622 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790304178325258-13942-0/minicluster-data/master-0/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790304178325258-13942-0/minicluster-data/master-0/wal
uuid: "3d0d39a173b54b1eb89375b7670be95f"
format_stamp: "Formatted at 2026-09-25 02:43:24 on dist-test-slave-kznw"
server_key: "b4f9feb7abc147ba97f4507db4736d42"
server_key_iv: "4f93ea37b60ee683856bdd774653d2a1"
server_key_version: "encryptionkey@0"
I20260925 02:43:24.023674 18622 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790304178325258-13942-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790304178325258-13942-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790304178325258-13942-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:43:24.028263 18622 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:43:24.028869 18622 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:43:24.029026 18622 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:43:24.035723 18622 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.190:45399
I20260925 02:43:24.035748 18689 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.190:45399 every 8 connection(s)
I20260925 02:43:24.036635 18622 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
I20260925 02:43:24.040879 18690 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:43:24.041509 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 18622
I20260925 02:43:24.041635 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790304178325258-13942-0/minicluster-data/master-0/wal/instance
I20260925 02:43:24.041904 13942 external_mini_cluster.cc:1428] Setting key 9ed3d49d81eb6d90bdde7a579e594768
I20260925 02:43:24.047072 18690 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d0d39a173b54b1eb89375b7670be95f: Bootstrap starting.
I20260925 02:43:24.047593 18693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:24.043428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58136 (local address 127.13.157.190:45399)
0925 02:43:24.044062 (+ 634us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:24.044078 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:24.044114 (+ 36us) server_negotiation.cc:407] Connection header received
0925 02:43:24.044766 (+ 652us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:24.044783 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:24.045144 (+ 361us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:24.045415 (+ 271us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:24.046241 (+ 826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:24.046246 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:24.046259 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:43:24.046270 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:24.046325 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:43:24.046341 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:24.046456 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:24.046552 (+ 96us) server_negotiation.cc:299] Negotiation successful
0925 02:43:24.046782 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":114,"threads_started":1}
I20260925 02:43:24.049469 18690 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3d0d39a173b54b1eb89375b7670be95f: Neither blocks nor log segments found. Creating new log.
I20260925 02:43:24.050264 18690 log.cc:824] T 00000000000000000000000000000000 P 3d0d39a173b54b1eb89375b7670be95f: Log is configured to *not* fsync() on all Append() calls
I20260925 02:43:24.052248 18690 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d0d39a173b54b1eb89375b7670be95f: No bootstrap required, opened a new log
I20260925 02:43:24.055730 18690 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3d0d39a173b54b1eb89375b7670be95f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d0d39a173b54b1eb89375b7670be95f" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 45399 } }
I20260925 02:43:24.055968 18690 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3d0d39a173b54b1eb89375b7670be95f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:43:24.056025 18690 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3d0d39a173b54b1eb89375b7670be95f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d0d39a173b54b1eb89375b7670be95f, State: Initialized, Role: FOLLOWER
I20260925 02:43:24.056452 18690 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3d0d39a173b54b1eb89375b7670be95f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d0d39a173b54b1eb89375b7670be95f" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 45399 } }
I20260925 02:43:24.056596 18690 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3d0d39a173b54b1eb89375b7670be95f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 02:43:24.056661 18690 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3d0d39a173b54b1eb89375b7670be95f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 02:43:24.056766 18690 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3d0d39a173b54b1eb89375b7670be95f [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:43:24.057621 18690 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3d0d39a173b54b1eb89375b7670be95f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d0d39a173b54b1eb89375b7670be95f" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 45399 } }
I20260925 02:43:24.057927 18690 leader_election.cc:302] T 00000000000000000000000000000000 P 3d0d39a173b54b1eb89375b7670be95f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3d0d39a173b54b1eb89375b7670be95f; no voters:
I20260925 02:43:24.058175 18690 leader_election.cc:288] T 00000000000000000000000000000000 P 3d0d39a173b54b1eb89375b7670be95f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 02:43:24.058286 18695 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3d0d39a173b54b1eb89375b7670be95f [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:43:24.058485 18695 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3d0d39a173b54b1eb89375b7670be95f [term 1 LEADER]: Becoming Leader. State: Replica: 3d0d39a173b54b1eb89375b7670be95f, State: Running, Role: LEADER
I20260925 02:43:24.058810 18695 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3d0d39a173b54b1eb89375b7670be95f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d0d39a173b54b1eb89375b7670be95f" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 45399 } }
I20260925 02:43:24.059376 18690 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3d0d39a173b54b1eb89375b7670be95f [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:43:24.060644 18697 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d0d39a173b54b1eb89375b7670be95f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3d0d39a173b54b1eb89375b7670be95f. Latest consensus state: current_term: 1 leader_uuid: "3d0d39a173b54b1eb89375b7670be95f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d0d39a173b54b1eb89375b7670be95f" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 45399 } } }
I20260925 02:43:24.060770 18697 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d0d39a173b54b1eb89375b7670be95f [sys.catalog]: This master's current role is: LEADER
I20260925 02:43:24.061067 18696 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d0d39a173b54b1eb89375b7670be95f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3d0d39a173b54b1eb89375b7670be95f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d0d39a173b54b1eb89375b7670be95f" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 45399 } } }
I20260925 02:43:24.061156 18696 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d0d39a173b54b1eb89375b7670be95f [sys.catalog]: This master's current role is: LEADER
W20260925 02:43:24.064160 18711 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3d0d39a173b54b1eb89375b7670be95f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 02:43:24.064229 18711 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 02:43:24.064285 18704 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:43:24.064942 18704 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:43:24.071331 18704 catalog_manager.cc:1347] Generated new cluster ID: 7762aacdcfaf4027b219b11474a8f02c
I20260925 02:43:24.071393 18704 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 02:43:24.092279 18704 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 02:43:24.092602 18704 catalog_manager.cc:1504] Loading token signing keys...
I20260925 02:43:24.099130 18704 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3d0d39a173b54b1eb89375b7670be95f: Generated new TSK 0
I20260925 02:43:24.099927 18704 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 02:43:24.103714 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790304178325258-13942-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790304178325258-13942-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790304178325258-13942-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790304178325258-13942-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.157.129:0
--local_ip_for_outbound_sockets=127.13.157.129
--webserver_interface=127.13.157.129
--webserver_port=0
--tserver_master_addrs=127.13.157.190:45399
--builtin_ntp_servers=127.13.157.148:38553
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260925 02:43:24.216675 18714 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:43:24.216943 18714 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:43:24.216984 18714 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:43:24.221443 18714 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:43:24.221510 18714 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:43:24.221568 18714 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.157.129
I20260925 02:43:24.226379 18714 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:38553
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790304178325258-13942-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790304178325258-13942-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.157.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790304178325258-13942-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.157.129
--webserver_port=0
--tserver_master_addrs=127.13.157.190:45399
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18714
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.157.129
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790304178325258-13942-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:43:24.227576 18714 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:43:24.229034 18714 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:43:24.238724 18719 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:24.238794 18720 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:43:24.238793 18714 server_base.cc:1034] running on GCE node
W20260925 02:43:24.238727 18722 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:43:24.239415 18714 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:43:24.240088 18714 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:43:24.241263 18714 hybrid_clock.cc:648] HybridClock initialized: now 1790304204241238 us; error 36 us; skew 500 ppm
I20260925 02:43:24.246888 18714 webserver.cc:460] Webserver started at http://127.13.157.129:46757/ using document root <none> and password file <none>
I20260925 02:43:24.247583 18714 fs_manager.cc:359] Metadata directory not provided
I20260925 02:43:24.247661 18714 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:43:24.247861 18714 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:43:24.250064 18714 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790304178325258-13942-0/minicluster-data/ts-0/data/instance:
uuid: "5ee162eade3547939f18f5b0e69e4c77"
format_stamp: "Formatted at 2026-09-25 02:43:24 on dist-test-slave-kznw"
server_key: "67dfcdd59fe24cad00fc85b5842a0e35"
server_key_iv: "068ed03c31c24b2208db9e185edc7b0d"
server_key_version: "encryptionkey@0"
I20260925 02:43:24.250617 18714 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790304178325258-13942-0/minicluster-data/ts-0/wal/instance:
uuid: "5ee162eade3547939f18f5b0e69e4c77"
format_stamp: "Formatted at 2026-09-25 02:43:24 on dist-test-slave-kznw"
server_key: "67dfcdd59fe24cad00fc85b5842a0e35"
server_key_iv: "068ed03c31c24b2208db9e185edc7b0d"
server_key_version: "encryptionkey@0"
I20260925 02:43:24.254349 18714 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.003s
I20260925 02:43:24.257169 18728 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:24.258457 18714 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260925 02:43:24.258615 18714 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790304178325258-13942-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790304178325258-13942-0/minicluster-data/ts-0/wal
uuid: "5ee162eade3547939f18f5b0e69e4c77"
format_stamp: "Formatted at 2026-09-25 02:43:24 on dist-test-slave-kznw"
server_key: "67dfcdd59fe24cad00fc85b5842a0e35"
server_key_iv: "068ed03c31c24b2208db9e185edc7b0d"
server_key_version: "encryptionkey@0"
I20260925 02:43:24.258729 18714 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790304178325258-13942-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790304178325258-13942-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790304178325258-13942-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:43:24.263389 18714 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:43:24.263988 18714 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:43:24.264144 18714 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:43:24.264674 18714 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:43:24.265604 18714 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:43:24.265651 18714 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:24.265703 18714 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:43:24.265733 18714 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:24.274328 18714 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.129:43591
I20260925 02:43:24.274407 18844 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.129:43591 every 8 connection(s)
I20260925 02:43:24.275306 18714 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790304178325258-13942-0/minicluster-data/ts-0/data/info.pb
I20260925 02:43:24.279870 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 18714
I20260925 02:43:24.279984 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790304178325258-13942-0/minicluster-data/ts-0/wal/instance
I20260925 02:43:24.280270 13942 external_mini_cluster.cc:1428] Setting key 4df5e7ffb5c866872ad6af9fae00241f
I20260925 02:43:24.281488 18693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:24.277132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.129:55987 (local address 127.13.157.190:45399)
0925 02:43:24.277260 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:24.277264 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:24.278293 (+ 1029us) server_negotiation.cc:407] Connection header received
0925 02:43:24.279667 (+ 1374us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:24.279671 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:24.279734 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:24.279809 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:24.281010 (+ 1201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:24.281014 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:24.281017 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:43:24.281034 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:24.281066 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:43:24.281077 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:24.281165 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:24.281306 (+ 141us) server_negotiation.cc:299] Negotiation successful
0925 02:43:24.281352 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260925 02:43:24.282568 18847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:24.277479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:45399 (local address 127.13.157.129:55987)
0925 02:43:24.278145 (+ 666us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:24.278186 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:24.279426 (+ 1240us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:24.279988 (+ 562us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:24.279997 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:24.280104 (+ 107us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:24.280654 (+ 550us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 02:43:24.280657 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:24.280709 (+ 52us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 02:43:24.280797 (+ 88us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:43:24.281167 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:43:24.281177 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:43:24.281184 (+ 7us) client_negotiation.cc:769] Sending connection context
0925 02:43:24.281345 (+ 161us) client_negotiation.cc:240] Negotiation successful
0925 02:43:24.281597 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":107,"threads_started":1}
I20260925 02:43:24.282572 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790304178325258-13942-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790304178325258-13942-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790304178325258-13942-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790304178325258-13942-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.157.130:0
--local_ip_for_outbound_sockets=127.13.157.130
--webserver_interface=127.13.157.130
--webserver_port=0
--tserver_master_addrs=127.13.157.190:45399
--builtin_ntp_servers=127.13.157.148:38553
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260925 02:43:24.283797 18845 heartbeater.cc:347] Connected to a master server at 127.13.157.190:45399
I20260925 02:43:24.284053 18845 heartbeater.cc:464] Registering TS with master...
I20260925 02:43:24.284512 18845 heartbeater.cc:511] Master 127.13.157.190:45399 requested a full tablet report, sending...
I20260925 02:43:24.286159 18654 ts_manager.cc:194] Registered new tserver with Master: 5ee162eade3547939f18f5b0e69e4c77 (127.13.157.129:43591)
W20260925 02:43:24.395040 18848 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:43:24.395318 18848 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:43:24.395358 18848 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:43:24.399516 18848 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:43:24.399574 18848 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:43:24.399641 18848 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.157.130
I20260925 02:43:24.404682 18848 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:38553
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790304178325258-13942-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790304178325258-13942-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.157.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790304178325258-13942-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.157.130
--webserver_port=0
--tserver_master_addrs=127.13.157.190:45399
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18848
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.157.130
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790304178325258-13942-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:43:24.405792 18848 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:43:24.407236 18848 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:43:24.416738 18853 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:24.416756 18856 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:24.416738 18854 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:43:24.416882 18848 server_base.cc:1034] running on GCE node
I20260925 02:43:24.417507 18848 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:43:24.418179 18848 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:43:24.419333 18848 hybrid_clock.cc:648] HybridClock initialized: now 1790304204419312 us; error 37 us; skew 500 ppm
I20260925 02:43:24.421211 18848 webserver.cc:460] Webserver started at http://127.13.157.130:44015/ using document root <none> and password file <none>
I20260925 02:43:24.421814 18848 fs_manager.cc:359] Metadata directory not provided
I20260925 02:43:24.421878 18848 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:43:24.422067 18848 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:43:24.424187 18848 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790304178325258-13942-0/minicluster-data/ts-1/data/instance:
uuid: "76226f1fcfb5419cad9e55fd4153d68d"
format_stamp: "Formatted at 2026-09-25 02:43:24 on dist-test-slave-kznw"
server_key: "75d83e304c2e3b4924d882e0e49a3d28"
server_key_iv: "969a8ebc91f596acb0b6259b9ebbbd36"
server_key_version: "encryptionkey@0"
I20260925 02:43:24.424744 18848 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790304178325258-13942-0/minicluster-data/ts-1/wal/instance:
uuid: "76226f1fcfb5419cad9e55fd4153d68d"
format_stamp: "Formatted at 2026-09-25 02:43:24 on dist-test-slave-kznw"
server_key: "75d83e304c2e3b4924d882e0e49a3d28"
server_key_iv: "969a8ebc91f596acb0b6259b9ebbbd36"
server_key_version: "encryptionkey@0"
I20260925 02:43:24.428510 18848 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260925 02:43:24.431169 18862 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:24.432361 18848 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260925 02:43:24.432461 18848 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790304178325258-13942-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790304178325258-13942-0/minicluster-data/ts-1/wal
uuid: "76226f1fcfb5419cad9e55fd4153d68d"
format_stamp: "Formatted at 2026-09-25 02:43:24 on dist-test-slave-kznw"
server_key: "75d83e304c2e3b4924d882e0e49a3d28"
server_key_iv: "969a8ebc91f596acb0b6259b9ebbbd36"
server_key_version: "encryptionkey@0"
I20260925 02:43:24.432569 18848 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790304178325258-13942-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790304178325258-13942-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790304178325258-13942-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:43:24.437096 18848 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:43:24.437690 18848 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:43:24.437839 18848 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:43:24.438359 18848 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:43:24.439301 18848 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:43:24.439347 18848 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:24.439383 18848 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:43:24.439412 18848 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:24.447638 18848 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.130:43425
I20260925 02:43:24.447655 18977 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.130:43425 every 8 connection(s)
I20260925 02:43:24.448544 18848 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790304178325258-13942-0/minicluster-data/ts-1/data/info.pb
I20260925 02:43:24.449283 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 18848
I20260925 02:43:24.449394 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790304178325258-13942-0/minicluster-data/ts-1/wal/instance
I20260925 02:43:24.449669 13942 external_mini_cluster.cc:1428] Setting key 5ff2141a660411630ef2a8caceb01702
I20260925 02:43:24.451834 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790304178325258-13942-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790304178325258-13942-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790304178325258-13942-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790304178325258-13942-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.157.131:0
--local_ip_for_outbound_sockets=127.13.157.131
--webserver_interface=127.13.157.131
--webserver_port=0
--tserver_master_addrs=127.13.157.190:45399
--builtin_ntp_servers=127.13.157.148:38553
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260925 02:43:24.454610 18693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:24.450559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.130:42451 (local address 127.13.157.190:45399)
0925 02:43:24.450708 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:24.450713 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:24.451730 (+ 1017us) server_negotiation.cc:407] Connection header received
0925 02:43:24.453020 (+ 1290us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:24.453023 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:24.453067 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:24.453140 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:24.454158 (+ 1018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:24.454162 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:24.454164 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:43:24.454177 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:24.454200 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:43:24.454208 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:24.454285 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:24.454409 (+ 124us) server_negotiation.cc:299] Negotiation successful
0925 02:43:24.454468 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260925 02:43:24.455526 18980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:24.450861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:45399 (local address 127.13.157.130:42451)
0925 02:43:24.451590 (+ 729us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:24.451636 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:24.452824 (+ 1188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:24.453304 (+ 480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:24.453311 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:24.453396 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:24.453909 (+ 513us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 02:43:24.453910 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:24.453949 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 02:43:24.454017 (+ 68us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:43:24.454275 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:43:24.454283 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:43:24.454289 (+ 6us) client_negotiation.cc:769] Sending connection context
0925 02:43:24.454443 (+ 154us) client_negotiation.cc:240] Negotiation successful
0925 02:43:24.454679 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":135,"threads_started":1}
I20260925 02:43:24.456691 18978 heartbeater.cc:347] Connected to a master server at 127.13.157.190:45399
I20260925 02:43:24.456940 18978 heartbeater.cc:464] Registering TS with master...
I20260925 02:43:24.457346 18978 heartbeater.cc:511] Master 127.13.157.190:45399 requested a full tablet report, sending...
I20260925 02:43:24.458390 18654 ts_manager.cc:194] Registered new tserver with Master: 76226f1fcfb5419cad9e55fd4153d68d (127.13.157.130:43425)
W20260925 02:43:24.562076 18981 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:43:24.562348 18981 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:43:24.562405 18981 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:43:24.566601 18981 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:43:24.566665 18981 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:43:24.566746 18981 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.157.131
I20260925 02:43:24.571614 18981 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:38553
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790304178325258-13942-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790304178325258-13942-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.157.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790304178325258-13942-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.157.131
--webserver_port=0
--tserver_master_addrs=127.13.157.190:45399
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18981
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.157.131
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790304178325258-13942-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:43:24.572728 18981 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:43:24.574451 18981 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:43:24.583765 18989 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:24.583966 18987 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:24.584015 18986 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:43:24.584106 18981 server_base.cc:1034] running on GCE node
I20260925 02:43:24.584477 18981 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:43:24.585120 18981 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:43:24.586292 18981 hybrid_clock.cc:648] HybridClock initialized: now 1790304204586270 us; error 31 us; skew 500 ppm
I20260925 02:43:24.588160 18981 webserver.cc:460] Webserver started at http://127.13.157.131:37813/ using document root <none> and password file <none>
I20260925 02:43:24.588714 18981 fs_manager.cc:359] Metadata directory not provided
I20260925 02:43:24.588773 18981 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:43:24.588958 18981 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:43:24.591068 18981 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790304178325258-13942-0/minicluster-data/ts-2/data/instance:
uuid: "040257a51e734317aeac094e9c1eb470"
format_stamp: "Formatted at 2026-09-25 02:43:24 on dist-test-slave-kznw"
server_key: "de169b3bfe3cfdeabb2540632f4442be"
server_key_iv: "0948fd7909323b405a946017fbae0753"
server_key_version: "encryptionkey@0"
I20260925 02:43:24.591671 18981 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790304178325258-13942-0/minicluster-data/ts-2/wal/instance:
uuid: "040257a51e734317aeac094e9c1eb470"
format_stamp: "Formatted at 2026-09-25 02:43:24 on dist-test-slave-kznw"
server_key: "de169b3bfe3cfdeabb2540632f4442be"
server_key_iv: "0948fd7909323b405a946017fbae0753"
server_key_version: "encryptionkey@0"
I20260925 02:43:24.595410 18981 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260925 02:43:24.597922 18996 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:24.599207 18981 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260925 02:43:24.599323 18981 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790304178325258-13942-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790304178325258-13942-0/minicluster-data/ts-2/wal
uuid: "040257a51e734317aeac094e9c1eb470"
format_stamp: "Formatted at 2026-09-25 02:43:24 on dist-test-slave-kznw"
server_key: "de169b3bfe3cfdeabb2540632f4442be"
server_key_iv: "0948fd7909323b405a946017fbae0753"
server_key_version: "encryptionkey@0"
I20260925 02:43:24.599428 18981 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790304178325258-13942-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790304178325258-13942-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790304178325258-13942-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:43:24.603888 18981 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:43:24.604446 18981 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:43:24.604573 18981 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:43:24.605050 18981 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:43:24.605897 18981 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:43:24.605940 18981 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:24.605976 18981 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:43:24.605996 18981 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:24.614494 18981 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.131:41965
I20260925 02:43:24.614516 19112 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.131:41965 every 8 connection(s)
I20260925 02:43:24.615460 18981 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790304178325258-13942-0/minicluster-data/ts-2/data/info.pb
I20260925 02:43:24.617831 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 18981
I20260925 02:43:24.617954 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790304178325258-13942-0/minicluster-data/ts-2/wal/instance
I20260925 02:43:24.618248 13942 external_mini_cluster.cc:1428] Setting key f43cb111d416d7c0910f6a49056e6894
I20260925 02:43:24.622028 18693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:24.617219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.131:42321 (local address 127.13.157.190:45399)
0925 02:43:24.617340 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:24.617343 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:24.618561 (+ 1218us) server_negotiation.cc:407] Connection header received
0925 02:43:24.620325 (+ 1764us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:24.620329 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:24.620375 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:24.620447 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:24.621576 (+ 1129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:24.621579 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:24.621582 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:43:24.621595 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:24.621616 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:43:24.621624 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:24.621698 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:24.621877 (+ 179us) server_negotiation.cc:299] Negotiation successful
0925 02:43:24.621912 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260925 02:43:24.623031 19115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:24.617630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:45399 (local address 127.13.157.131:42321)
0925 02:43:24.618414 (+ 784us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:24.618454 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:24.620075 (+ 1621us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:24.620630 (+ 555us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:24.620637 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:24.620729 (+ 92us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:24.621298 (+ 569us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 02:43:24.621300 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:24.621341 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 02:43:24.621413 (+ 72us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:43:24.621751 (+ 338us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:43:24.621760 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:43:24.621765 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 02:43:24.621921 (+ 156us) client_negotiation.cc:240] Negotiation successful
0925 02:43:24.622199 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":147,"threads_started":1}
I20260925 02:43:24.624115 19113 heartbeater.cc:347] Connected to a master server at 127.13.157.190:45399
I20260925 02:43:24.624325 19113 heartbeater.cc:464] Registering TS with master...
I20260925 02:43:24.624706 19113 heartbeater.cc:511] Master 127.13.157.190:45399 requested a full tablet report, sending...
I20260925 02:43:24.625756 18654 ts_manager.cc:194] Registered new tserver with Master: 040257a51e734317aeac094e9c1eb470 (127.13.157.131:41965)
I20260925 02:43:24.632251 13942 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 02:43:24.635490 18693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:24.633956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58150 (local address 127.13.157.190:45399)
0925 02:43:24.634080 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:24.634085 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:24.634248 (+ 163us) server_negotiation.cc:407] Connection header received
0925 02:43:24.634359 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:24.634362 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:24.634397 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:24.634457 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:24.635128 (+ 671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:24.635131 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:24.635134 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:43:24.635145 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:24.635165 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:43:24.635175 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:24.635244 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:24.635325 (+ 81us) server_negotiation.cc:299] Negotiation successful
0925 02:43:24.635364 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260925 02:43:24.638370 18654 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58150:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 02:43:24.641480 18654 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 02:43:24.653117 19130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:24.651152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34376 (local address 127.13.157.131:41965)
0925 02:43:24.651443 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:24.651454 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:24.651478 (+ 24us) server_negotiation.cc:407] Connection header received
0925 02:43:24.651528 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:24.651533 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:24.651713 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:24.651909 (+ 196us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:24.652601 (+ 692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:24.652606 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:24.652616 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:43:24.652625 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:24.652657 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:43:24.652674 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:24.652766 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:24.652894 (+ 128us) server_negotiation.cc:299] Negotiation successful
0925 02:43:24.652967 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":200,"thread_start_us":96,"threads_started":1}
I20260925 02:43:24.653127 19124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:24.649988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.130:43425 (local address 127.0.0.1:54252)
0925 02:43:24.650369 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:24.650421 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:24.650525 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:24.651518 (+ 993us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:24.651524 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:24.651603 (+ 79us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:24.652231 (+ 628us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 02:43:24.652232 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:24.652252 (+ 20us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 02:43:24.652322 (+ 70us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:43:24.652798 (+ 476us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:43:24.652801 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:43:24.652804 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 02:43:24.652861 (+ 57us) client_negotiation.cc:240] Negotiation successful
0925 02:43:24.652933 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":101,"threads_started":1}
I20260925 02:43:24.653105 19127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:24.650731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.131:41965 (local address 127.0.0.1:34376)
0925 02:43:24.651076 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:24.651089 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:24.651165 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:24.651858 (+ 693us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:24.651862 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:24.651877 (+ 15us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:24.652449 (+ 572us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 02:43:24.652451 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:24.652460 (+ 9us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 02:43:24.652479 (+ 19us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:43:24.652751 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:43:24.652761 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:43:24.652766 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 02:43:24.652873 (+ 107us) client_negotiation.cc:240] Negotiation successful
0925 02:43:24.652937 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":88,"threads_started":1}
I20260925 02:43:24.653599 19125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:24.650132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54252 (local address 127.13.157.130:43425)
0925 02:43:24.651056 (+ 924us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:24.651068 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:24.651092 (+ 24us) server_negotiation.cc:407] Connection header received
0925 02:43:24.651145 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:24.651151 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:24.651349 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:24.651911 (+ 562us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:24.652605 (+ 694us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:24.652610 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:24.652621 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:43:24.652632 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:24.652671 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:43:24.652695 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:24.653342 (+ 647us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:24.653383 (+ 41us) server_negotiation.cc:299] Negotiation successful
0925 02:43:24.653446 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":820,"thread_start_us":146,"threads_started":1}
I20260925 02:43:24.654426 19129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:24.650693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38200 (local address 127.13.157.129:43591)
0925 02:43:24.651167 (+ 474us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:24.651182 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:24.651200 (+ 18us) server_negotiation.cc:407] Connection header received
0925 02:43:24.651243 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:24.651247 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:24.651371 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:24.652638 (+ 1267us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:24.653194 (+ 556us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:24.653199 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:24.653410 (+ 211us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:43:24.653433 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:24.653474 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:43:24.653506 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:24.654180 (+ 674us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:24.654226 (+ 46us) server_negotiation.cc:299] Negotiation successful
0925 02:43:24.654288 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389,"thread_start_us":131,"threads_started":1}
I20260925 02:43:24.654814 19126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:24.650216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.129:43591 (local address 127.0.0.1:38200)
0925 02:43:24.650726 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:24.650738 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:24.650805 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:24.651518 (+ 713us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:24.651524 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:24.651603 (+ 79us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:24.652453 (+ 850us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 02:43:24.652455 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:24.652467 (+ 12us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 02:43:24.652497 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:43:24.653613 (+ 1116us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:43:24.653616 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:43:24.653620 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 02:43:24.654661 (+ 1041us) client_negotiation.cc:240] Negotiation successful
0925 02:43:24.654703 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":116,"threads_started":1}
I20260925 02:43:24.656188 18910 tablet_service.cc:1467] Processing CreateTablet for tablet 220f014dfe4f4a66a90189d76df90134 (DEFAULT_TABLE table=test-table [id=54aa193d9aa54425acb63cde843c1446]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:43:24.656231 19045 tablet_service.cc:1467] Processing CreateTablet for tablet 220f014dfe4f4a66a90189d76df90134 (DEFAULT_TABLE table=test-table [id=54aa193d9aa54425acb63cde843c1446]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:43:24.657164 18910 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 220f014dfe4f4a66a90189d76df90134. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:43:24.657256 19045 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 220f014dfe4f4a66a90189d76df90134. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:43:24.657398 18777 tablet_service.cc:1467] Processing CreateTablet for tablet 220f014dfe4f4a66a90189d76df90134 (DEFAULT_TABLE table=test-table [id=54aa193d9aa54425acb63cde843c1446]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:43:24.658342 18777 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 220f014dfe4f4a66a90189d76df90134. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:43:24.663447 19131 tablet_bootstrap.cc:492] T 220f014dfe4f4a66a90189d76df90134 P 76226f1fcfb5419cad9e55fd4153d68d: Bootstrap starting.
I20260925 02:43:24.665057 19132 tablet_bootstrap.cc:492] T 220f014dfe4f4a66a90189d76df90134 P 040257a51e734317aeac094e9c1eb470: Bootstrap starting.
I20260925 02:43:24.665900 19133 tablet_bootstrap.cc:492] T 220f014dfe4f4a66a90189d76df90134 P 5ee162eade3547939f18f5b0e69e4c77: Bootstrap starting.
I20260925 02:43:24.666211 19131 tablet_bootstrap.cc:654] T 220f014dfe4f4a66a90189d76df90134 P 76226f1fcfb5419cad9e55fd4153d68d: Neither blocks nor log segments found. Creating new log.
I20260925 02:43:24.667126 19131 log.cc:824] T 220f014dfe4f4a66a90189d76df90134 P 76226f1fcfb5419cad9e55fd4153d68d: Log is configured to *not* fsync() on all Append() calls
I20260925 02:43:24.667238 19132 tablet_bootstrap.cc:654] T 220f014dfe4f4a66a90189d76df90134 P 040257a51e734317aeac094e9c1eb470: Neither blocks nor log segments found. Creating new log.
I20260925 02:43:24.667944 19133 tablet_bootstrap.cc:654] T 220f014dfe4f4a66a90189d76df90134 P 5ee162eade3547939f18f5b0e69e4c77: Neither blocks nor log segments found. Creating new log.
I20260925 02:43:24.668004 19132 log.cc:824] T 220f014dfe4f4a66a90189d76df90134 P 040257a51e734317aeac094e9c1eb470: Log is configured to *not* fsync() on all Append() calls
I20260925 02:43:24.668648 19133 log.cc:824] T 220f014dfe4f4a66a90189d76df90134 P 5ee162eade3547939f18f5b0e69e4c77: Log is configured to *not* fsync() on all Append() calls
I20260925 02:43:24.669378 19131 tablet_bootstrap.cc:492] T 220f014dfe4f4a66a90189d76df90134 P 76226f1fcfb5419cad9e55fd4153d68d: No bootstrap required, opened a new log
I20260925 02:43:24.669595 19131 ts_tablet_manager.cc:1397] T 220f014dfe4f4a66a90189d76df90134 P 76226f1fcfb5419cad9e55fd4153d68d: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260925 02:43:24.669987 19132 tablet_bootstrap.cc:492] T 220f014dfe4f4a66a90189d76df90134 P 040257a51e734317aeac094e9c1eb470: No bootstrap required, opened a new log
I20260925 02:43:24.670176 19132 ts_tablet_manager.cc:1397] T 220f014dfe4f4a66a90189d76df90134 P 040257a51e734317aeac094e9c1eb470: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260925 02:43:24.670717 19133 tablet_bootstrap.cc:492] T 220f014dfe4f4a66a90189d76df90134 P 5ee162eade3547939f18f5b0e69e4c77: No bootstrap required, opened a new log
I20260925 02:43:24.670889 19133 ts_tablet_manager.cc:1397] T 220f014dfe4f4a66a90189d76df90134 P 5ee162eade3547939f18f5b0e69e4c77: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260925 02:43:24.674250 19133 raft_consensus.cc:348] T 220f014dfe4f4a66a90189d76df90134 P 5ee162eade3547939f18f5b0e69e4c77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76226f1fcfb5419cad9e55fd4153d68d" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 43425 } } peers { permanent_uuid: "5ee162eade3547939f18f5b0e69e4c77" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 43591 } } peers { permanent_uuid: "040257a51e734317aeac094e9c1eb470" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 41965 } }
I20260925 02:43:24.674270 19131 raft_consensus.cc:348] T 220f014dfe4f4a66a90189d76df90134 P 76226f1fcfb5419cad9e55fd4153d68d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76226f1fcfb5419cad9e55fd4153d68d" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 43425 } } peers { permanent_uuid: "5ee162eade3547939f18f5b0e69e4c77" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 43591 } } peers { permanent_uuid: "040257a51e734317aeac094e9c1eb470" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 41965 } }
I20260925 02:43:24.674494 19133 raft_consensus.cc:374] T 220f014dfe4f4a66a90189d76df90134 P 5ee162eade3547939f18f5b0e69e4c77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:43:24.674531 19131 raft_consensus.cc:374] T 220f014dfe4f4a66a90189d76df90134 P 76226f1fcfb5419cad9e55fd4153d68d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:43:24.674563 19133 raft_consensus.cc:729] T 220f014dfe4f4a66a90189d76df90134 P 5ee162eade3547939f18f5b0e69e4c77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ee162eade3547939f18f5b0e69e4c77, State: Initialized, Role: FOLLOWER
I20260925 02:43:24.674587 19131 raft_consensus.cc:729] T 220f014dfe4f4a66a90189d76df90134 P 76226f1fcfb5419cad9e55fd4153d68d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76226f1fcfb5419cad9e55fd4153d68d, State: Initialized, Role: FOLLOWER
I20260925 02:43:24.674439 19132 raft_consensus.cc:348] T 220f014dfe4f4a66a90189d76df90134 P 040257a51e734317aeac094e9c1eb470 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76226f1fcfb5419cad9e55fd4153d68d" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 43425 } } peers { permanent_uuid: "5ee162eade3547939f18f5b0e69e4c77" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 43591 } } peers { permanent_uuid: "040257a51e734317aeac094e9c1eb470" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 41965 } }
I20260925 02:43:24.674657 19132 raft_consensus.cc:374] T 220f014dfe4f4a66a90189d76df90134 P 040257a51e734317aeac094e9c1eb470 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:43:24.674708 19132 raft_consensus.cc:729] T 220f014dfe4f4a66a90189d76df90134 P 040257a51e734317aeac094e9c1eb470 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 040257a51e734317aeac094e9c1eb470, State: Initialized, Role: FOLLOWER
I20260925 02:43:24.675129 19131 consensus_queue.cc:260] T 220f014dfe4f4a66a90189d76df90134 P 76226f1fcfb5419cad9e55fd4153d68d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76226f1fcfb5419cad9e55fd4153d68d" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 43425 } } peers { permanent_uuid: "5ee162eade3547939f18f5b0e69e4c77" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 43591 } } peers { permanent_uuid: "040257a51e734317aeac094e9c1eb470" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 41965 } }
I20260925 02:43:24.675129 19132 consensus_queue.cc:260] T 220f014dfe4f4a66a90189d76df90134 P 040257a51e734317aeac094e9c1eb470 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76226f1fcfb5419cad9e55fd4153d68d" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 43425 } } peers { permanent_uuid: "5ee162eade3547939f18f5b0e69e4c77" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 43591 } } peers { permanent_uuid: "040257a51e734317aeac094e9c1eb470" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 41965 } }
I20260925 02:43:24.675130 19133 consensus_queue.cc:260] T 220f014dfe4f4a66a90189d76df90134 P 5ee162eade3547939f18f5b0e69e4c77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76226f1fcfb5419cad9e55fd4153d68d" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 43425 } } peers { permanent_uuid: "5ee162eade3547939f18f5b0e69e4c77" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 43591 } } peers { permanent_uuid: "040257a51e734317aeac094e9c1eb470" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 41965 } }
I20260925 02:43:24.675706 19137 raft_consensus.cc:482] T 220f014dfe4f4a66a90189d76df90134 P 5ee162eade3547939f18f5b0e69e4c77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 02:43:24.675889 19137 raft_consensus.cc:504] T 220f014dfe4f4a66a90189d76df90134 P 5ee162eade3547939f18f5b0e69e4c77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76226f1fcfb5419cad9e55fd4153d68d" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 43425 } } peers { permanent_uuid: "5ee162eade3547939f18f5b0e69e4c77" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 43591 } } peers { permanent_uuid: "040257a51e734317aeac094e9c1eb470" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 41965 } }
I20260925 02:43:24.676174 19132 ts_tablet_manager.cc:1428] T 220f014dfe4f4a66a90189d76df90134 P 040257a51e734317aeac094e9c1eb470: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260925 02:43:24.676174 19131 ts_tablet_manager.cc:1428] T 220f014dfe4f4a66a90189d76df90134 P 76226f1fcfb5419cad9e55fd4153d68d: Time spent starting tablet: real 0.007s user 0.004s sys 0.004s
I20260925 02:43:24.676532 19113 heartbeater.cc:503] Master 127.13.157.190:45399 was elected leader, sending a full tablet report...
I20260925 02:43:24.676532 18978 heartbeater.cc:503] Master 127.13.157.190:45399 was elected leader, sending a full tablet report...
I20260925 02:43:24.676664 19133 ts_tablet_manager.cc:1428] T 220f014dfe4f4a66a90189d76df90134 P 5ee162eade3547939f18f5b0e69e4c77: Time spent starting tablet: real 0.006s user 0.002s sys 0.003s
I20260925 02:43:24.676998 18845 heartbeater.cc:503] Master 127.13.157.190:45399 was elected leader, sending a full tablet report...
I20260925 02:43:24.677385 19137 leader_election.cc:288] T 220f014dfe4f4a66a90189d76df90134 P 5ee162eade3547939f18f5b0e69e4c77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 76226f1fcfb5419cad9e55fd4153d68d (127.13.157.130:43425), 040257a51e734317aeac094e9c1eb470 (127.13.157.131:41965)
I20260925 02:43:24.679749 19125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:24.678274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.129:45753 (local address 127.13.157.130:43425)
0925 02:43:24.678379 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:24.678383 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:24.678396 (+ 13us) server_negotiation.cc:407] Connection header received
0925 02:43:24.678433 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:24.678436 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:24.678472 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:24.678544 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:24.679318 (+ 774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:24.679322 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:24.679325 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:43:24.679336 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:24.679359 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:43:24.679370 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:24.679448 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:24.679556 (+ 108us) server_negotiation.cc:299] Negotiation successful
0925 02:43:24.679602 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260925 02:43:24.679936 18847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:24.677719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.130:43425 (local address 127.13.157.129:45753)
0925 02:43:24.677972 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:24.677986 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:24.678055 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:24.678655 (+ 600us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:24.678659 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:24.678673 (+ 14us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:24.679165 (+ 492us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 02:43:24.679167 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:24.679179 (+ 12us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 02:43:24.679209 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:43:24.679465 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:43:24.679468 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:43:24.679472 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 02:43:24.679545 (+ 73us) client_negotiation.cc:240] Negotiation successful
0925 02:43:24.679829 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173}
I20260925 02:43:24.680680 18930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "220f014dfe4f4a66a90189d76df90134" candidate_uuid: "5ee162eade3547939f18f5b0e69e4c77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76226f1fcfb5419cad9e55fd4153d68d" is_pre_election: true
I20260925 02:43:24.680770 19130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:24.678484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.129:50433 (local address 127.13.157.131:41965)
0925 02:43:24.679213 (+ 729us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:24.679217 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:24.679231 (+ 14us) server_negotiation.cc:407] Connection header received
0925 02:43:24.679267 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:24.679270 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:24.679302 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:24.679376 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:24.680324 (+ 948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:24.680327 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:24.680329 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:43:24.680341 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:24.680361 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:43:24.680369 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:24.680426 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:24.680618 (+ 192us) server_negotiation.cc:299] Negotiation successful
0925 02:43:24.680663 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314}
I20260925 02:43:24.680763 19141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:24.677975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.131:41965 (local address 127.13.157.129:50433)
0925 02:43:24.678649 (+ 674us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:24.678661 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:24.678724 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:24.679418 (+ 694us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:24.679420 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:24.679436 (+ 16us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:24.680149 (+ 713us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 02:43:24.680151 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:24.680162 (+ 11us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 02:43:24.680190 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:43:24.680495 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:43:24.680499 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:43:24.680501 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 02:43:24.680598 (+ 97us) client_negotiation.cc:240] Negotiation successful
0925 02:43:24.680632 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":613,"thread_start_us":100,"threads_started":1}
I20260925 02:43:24.680970 18930 raft_consensus.cc:2454] T 220f014dfe4f4a66a90189d76df90134 P 76226f1fcfb5419cad9e55fd4153d68d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ee162eade3547939f18f5b0e69e4c77 in term 0.
I20260925 02:43:24.681591 18730 leader_election.cc:302] T 220f014dfe4f4a66a90189d76df90134 P 5ee162eade3547939f18f5b0e69e4c77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ee162eade3547939f18f5b0e69e4c77, 76226f1fcfb5419cad9e55fd4153d68d; no voters:
I20260925 02:43:24.681644 19066 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "220f014dfe4f4a66a90189d76df90134" candidate_uuid: "5ee162eade3547939f18f5b0e69e4c77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "040257a51e734317aeac094e9c1eb470" is_pre_election: true
I20260925 02:43:24.681881 19066 raft_consensus.cc:2454] T 220f014dfe4f4a66a90189d76df90134 P 040257a51e734317aeac094e9c1eb470 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ee162eade3547939f18f5b0e69e4c77 in term 0.
I20260925 02:43:24.681897 19139 raft_consensus.cc:2781] T 220f014dfe4f4a66a90189d76df90134 P 5ee162eade3547939f18f5b0e69e4c77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 02:43:24.681981 19139 raft_consensus.cc:482] T 220f014dfe4f4a66a90189d76df90134 P 5ee162eade3547939f18f5b0e69e4c77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 02:43:24.682026 19139 raft_consensus.cc:3037] T 220f014dfe4f4a66a90189d76df90134 P 5ee162eade3547939f18f5b0e69e4c77 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:43:24.683431 19139 raft_consensus.cc:504] T 220f014dfe4f4a66a90189d76df90134 P 5ee162eade3547939f18f5b0e69e4c77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76226f1fcfb5419cad9e55fd4153d68d" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 43425 } } peers { permanent_uuid: "5ee162eade3547939f18f5b0e69e4c77" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 43591 } } peers { permanent_uuid: "040257a51e734317aeac094e9c1eb470" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 41965 } }
I20260925 02:43:24.683835 19139 leader_election.cc:288] T 220f014dfe4f4a66a90189d76df90134 P 5ee162eade3547939f18f5b0e69e4c77 [CANDIDATE]: Term 1 election: Requested vote from peers 76226f1fcfb5419cad9e55fd4153d68d (127.13.157.130:43425), 040257a51e734317aeac094e9c1eb470 (127.13.157.131:41965)
I20260925 02:43:24.684182 18930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "220f014dfe4f4a66a90189d76df90134" candidate_uuid: "5ee162eade3547939f18f5b0e69e4c77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76226f1fcfb5419cad9e55fd4153d68d"
I20260925 02:43:24.684300 18930 raft_consensus.cc:3037] T 220f014dfe4f4a66a90189d76df90134 P 76226f1fcfb5419cad9e55fd4153d68d [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:43:24.684365 19066 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "220f014dfe4f4a66a90189d76df90134" candidate_uuid: "5ee162eade3547939f18f5b0e69e4c77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "040257a51e734317aeac094e9c1eb470"
I20260925 02:43:24.684489 19066 raft_consensus.cc:3037] T 220f014dfe4f4a66a90189d76df90134 P 040257a51e734317aeac094e9c1eb470 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:43:24.685289 18930 raft_consensus.cc:2454] T 220f014dfe4f4a66a90189d76df90134 P 76226f1fcfb5419cad9e55fd4153d68d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ee162eade3547939f18f5b0e69e4c77 in term 1.
I20260925 02:43:24.685597 19066 raft_consensus.cc:2454] T 220f014dfe4f4a66a90189d76df90134 P 040257a51e734317aeac094e9c1eb470 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ee162eade3547939f18f5b0e69e4c77 in term 1.
I20260925 02:43:24.685591 18730 leader_election.cc:302] T 220f014dfe4f4a66a90189d76df90134 P 5ee162eade3547939f18f5b0e69e4c77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ee162eade3547939f18f5b0e69e4c77, 76226f1fcfb5419cad9e55fd4153d68d; no voters:
I20260925 02:43:24.685839 19139 raft_consensus.cc:2781] T 220f014dfe4f4a66a90189d76df90134 P 5ee162eade3547939f18f5b0e69e4c77 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:43:24.685954 19139 raft_consensus.cc:686] T 220f014dfe4f4a66a90189d76df90134 P 5ee162eade3547939f18f5b0e69e4c77 [term 1 LEADER]: Becoming Leader. State: Replica: 5ee162eade3547939f18f5b0e69e4c77, State: Running, Role: LEADER
I20260925 02:43:24.686321 19139 consensus_queue.cc:237] T 220f014dfe4f4a66a90189d76df90134 P 5ee162eade3547939f18f5b0e69e4c77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76226f1fcfb5419cad9e55fd4153d68d" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 43425 } } peers { permanent_uuid: "5ee162eade3547939f18f5b0e69e4c77" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 43591 } } peers { permanent_uuid: "040257a51e734317aeac094e9c1eb470" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 41965 } }
I20260925 02:43:24.689843 18654 catalog_manager.cc:5495] T 220f014dfe4f4a66a90189d76df90134 P 5ee162eade3547939f18f5b0e69e4c77 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5ee162eade3547939f18f5b0e69e4c77 (127.13.157.129). New cstate: current_term: 1 leader_uuid: "5ee162eade3547939f18f5b0e69e4c77" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76226f1fcfb5419cad9e55fd4153d68d" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 43425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5ee162eade3547939f18f5b0e69e4c77" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 43591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "040257a51e734317aeac094e9c1eb470" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 41965 } health_report { overall_health: UNKNOWN } } }
W20260925 02:43:24.699481 18979 tablet.cc:2369] T 220f014dfe4f4a66a90189d76df90134 P 76226f1fcfb5419cad9e55fd4153d68d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 02:43:24.708401 19129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:24.707143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38202 (local address 127.13.157.129:43591)
0925 02:43:24.707265 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:24.707269 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:24.707283 (+ 14us) server_negotiation.cc:407] Connection header received
0925 02:43:24.707380 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:24.707383 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:24.707443 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:24.707513 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:24.708105 (+ 592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:24.708108 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:24.708110 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:43:24.708122 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:24.708142 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:43:24.708150 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:24.708220 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:24.708262 (+ 42us) server_negotiation.cc:299] Negotiation successful
0925 02:43:24.708294 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260925 02:43:24.714408 18930 raft_consensus.cc:1261] T 220f014dfe4f4a66a90189d76df90134 P 76226f1fcfb5419cad9e55fd4153d68d [term 1 FOLLOWER]: Refusing update from remote peer 5ee162eade3547939f18f5b0e69e4c77: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:43:24.714408 19066 raft_consensus.cc:1261] T 220f014dfe4f4a66a90189d76df90134 P 040257a51e734317aeac094e9c1eb470 [term 1 FOLLOWER]: Refusing update from remote peer 5ee162eade3547939f18f5b0e69e4c77: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:43:24.715112 19137 consensus_queue.cc:1035] T 220f014dfe4f4a66a90189d76df90134 P 5ee162eade3547939f18f5b0e69e4c77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "040257a51e734317aeac094e9c1eb470" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 41965 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:43:24.715301 19139 consensus_queue.cc:1035] T 220f014dfe4f4a66a90189d76df90134 P 5ee162eade3547939f18f5b0e69e4c77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "76226f1fcfb5419cad9e55fd4153d68d" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 43425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:43:24.731282 18654 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:58150:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 02:43:24.731463 18654 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:58150:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 02:43:24.733822 18654 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3d0d39a173b54b1eb89375b7670be95f: Sending DeleteTablet for 3 replicas of tablet 220f014dfe4f4a66a90189d76df90134
I20260925 02:43:24.734661 18910 tablet_service.cc:1514] Processing DeleteTablet for tablet 220f014dfe4f4a66a90189d76df90134 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:43:24 UTC) from {username='slave'} at 127.0.0.1:54252
I20260925 02:43:24.734737 19045 tablet_service.cc:1514] Processing DeleteTablet for tablet 220f014dfe4f4a66a90189d76df90134 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:43:24 UTC) from {username='slave'} at 127.0.0.1:34376
I20260925 02:43:24.735211 18777 tablet_service.cc:1514] Processing DeleteTablet for tablet 220f014dfe4f4a66a90189d76df90134 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:43:24 UTC) from {username='slave'} at 127.0.0.1:38200
I20260925 02:43:24.735704 19156 tablet_replica.cc:330] T 220f014dfe4f4a66a90189d76df90134 P 5ee162eade3547939f18f5b0e69e4c77: stopping tablet replica
I20260925 02:43:24.735733 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 18714
I20260925 02:43:24.737342 19155 tablet_replica.cc:330] T 220f014dfe4f4a66a90189d76df90134 P 040257a51e734317aeac094e9c1eb470: stopping tablet replica
I20260925 02:43:24.737649 19155 raft_consensus.cc:2229] T 220f014dfe4f4a66a90189d76df90134 P 040257a51e734317aeac094e9c1eb470 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 02:43:24.737946 19155 raft_consensus.cc:2258] T 220f014dfe4f4a66a90189d76df90134 P 040257a51e734317aeac094e9c1eb470 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:43:24.738237 19154 tablet_replica.cc:330] T 220f014dfe4f4a66a90189d76df90134 P 76226f1fcfb5419cad9e55fd4153d68d: stopping tablet replica
I20260925 02:43:24.738549 19154 raft_consensus.cc:2229] T 220f014dfe4f4a66a90189d76df90134 P 76226f1fcfb5419cad9e55fd4153d68d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 02:43:24.738853 19154 raft_consensus.cc:2258] T 220f014dfe4f4a66a90189d76df90134 P 76226f1fcfb5419cad9e55fd4153d68d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:43:24.739709 19155 ts_tablet_manager.cc:1905] T 220f014dfe4f4a66a90189d76df90134 P 040257a51e734317aeac094e9c1eb470: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:43:24.741092 19154 ts_tablet_manager.cc:1905] T 220f014dfe4f4a66a90189d76df90134 P 76226f1fcfb5419cad9e55fd4153d68d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:43:24.742532 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 18848
W20260925 02:43:24.742520 18637 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.157.129:43591 (error 108)
I20260925 02:43:24.742564 19155 ts_tablet_manager.cc:1918] T 220f014dfe4f4a66a90189d76df90134 P 040257a51e734317aeac094e9c1eb470: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 02:43:24.742671 19155 log.cc:1196] T 220f014dfe4f4a66a90189d76df90134 P 040257a51e734317aeac094e9c1eb470: Deleting WAL directory at /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790304178325258-13942-0/minicluster-data/ts-2/wal/wals/220f014dfe4f4a66a90189d76df90134
I20260925 02:43:24.743098 19155 ts_tablet_manager.cc:1939] T 220f014dfe4f4a66a90189d76df90134 P 040257a51e734317aeac094e9c1eb470: Deleting consensus metadata
I20260925 02:43:24.744441 18637 catalog_manager.cc:4841] TS 040257a51e734317aeac094e9c1eb470 (127.13.157.131:41965): tablet 220f014dfe4f4a66a90189d76df90134 (table test-table [id=54aa193d9aa54425acb63cde843c1446]) successfully deleted
I20260925 02:43:24.747798 19127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:24.747470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.129:43591 (local address 127.0.0.1:38212)
0925 02:43:24.747635 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:24.747715 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.129:43591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260925 02:43:24.747978 18637 catalog_manager.cc:4584] TS 5ee162eade3547939f18f5b0e69e4c77 (127.13.157.129:43591): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 220f014dfe4f4a66a90189d76df90134: Network error: Client connection negotiation failed: client connection to 127.13.157.129:43591: connect: Connection refused (error 111)
I20260925 02:43:24.748445 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 18981
I20260925 02:43:24.750113 19127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:24.748491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.130:43425 (local address 127.0.0.1:54254)
0925 02:43:24.749955 (+ 1464us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:24.750025 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.130:43425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1381}
I20260925 02:43:24.753690 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 18622
2026-09-25T02:43:24Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (905 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260925 02:43:24.773970 13942 mini_oidc.cc:152] Starting JWKS server
I20260925 02:43:24.776484 13942 webserver.cc:460] Webserver started at https://127.0.0.1:33137/ using document root <none> and password file <none>
I20260925 02:43:24.776677 13942 mini_oidc.cc:189] Starting OIDC Discovery server
I20260925 02:43:24.776942 13942 webserver.cc:460] Webserver started at http://127.0.0.1:33395/ using document root <none> and password file <none>
2026-09-25T02:43:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T02:43:24Z Disabled control of system clock
I20260925 02:43:24.785277 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790304178325258-13942-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790304178325258-13942-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790304178325258-13942-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.157.190:37457
--webserver_interface=127.13.157.190
--webserver_port=0
--builtin_ntp_servers=127.13.157.148:43903
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.157.190:37457
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33137/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790304178325258-13942-0/testchainca.cert with env {}
W20260925 02:43:24.897894 19163 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:43:24.898200 19163 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:43:24.898241 19163 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:43:24.902858 19163 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:43:24.902947 19163 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:43:24.902963 19163 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:43:24.902978 19163 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:43:24.902992 19163 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260925 02:43:24.903007 19163 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260925 02:43:24.903019 19163 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:33137/jwks.json
I20260925 02:43:24.908576 19163 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:43903
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790304178325258-13942-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790304178325258-13942-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.157.190:37457
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.157.190:37457
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33137/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.157.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790304178325258-13942-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19163
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790304178325258-13942-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:43:24.909664 19163 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:43:24.910671 19163 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:43:24.919574 19170 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:24.919595 19168 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:43:24.919697 19163 server_base.cc:1034] running on GCE node
W20260925 02:43:24.919538 19172 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:43:24.920532 19163 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:43:24.921752 19163 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:43:24.922963 19163 hybrid_clock.cc:648] HybridClock initialized: now 1790304204922968 us; error 69 us; skew 500 ppm
I20260925 02:43:24.924926 19163 webserver.cc:460] Webserver started at http://127.13.157.190:41805/ using document root <none> and password file <none>
I20260925 02:43:24.925503 19163 fs_manager.cc:359] Metadata directory not provided
I20260925 02:43:24.925572 19163 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:43:24.925788 19163 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:43:24.927980 19163 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790304178325258-13942-0/minicluster-data/master-0/data/instance:
uuid: "b18187afb3104ee0a06222033ab61b9c"
format_stamp: "Formatted at 2026-09-25 02:43:24 on dist-test-slave-kznw"
server_key: "420c738ed20b91e2592061c13b2df98d"
server_key_iv: "cd0b84b3732d3dadd3ba173bfc147f0e"
server_key_version: "encryptionkey@0"
I20260925 02:43:24.928553 19163 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790304178325258-13942-0/minicluster-data/master-0/wal/instance:
uuid: "b18187afb3104ee0a06222033ab61b9c"
format_stamp: "Formatted at 2026-09-25 02:43:24 on dist-test-slave-kznw"
server_key: "420c738ed20b91e2592061c13b2df98d"
server_key_iv: "cd0b84b3732d3dadd3ba173bfc147f0e"
server_key_version: "encryptionkey@0"
I20260925 02:43:24.932277 19163 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260925 02:43:24.934686 19178 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:24.935956 19163 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260925 02:43:24.936071 19163 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790304178325258-13942-0/minicluster-data/master-0/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790304178325258-13942-0/minicluster-data/master-0/wal
uuid: "b18187afb3104ee0a06222033ab61b9c"
format_stamp: "Formatted at 2026-09-25 02:43:24 on dist-test-slave-kznw"
server_key: "420c738ed20b91e2592061c13b2df98d"
server_key_iv: "cd0b84b3732d3dadd3ba173bfc147f0e"
server_key_version: "encryptionkey@0"
I20260925 02:43:24.936187 19163 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790304178325258-13942-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790304178325258-13942-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790304178325258-13942-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:43:24.964561 19163 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:43:24.965327 19163 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:43:24.965509 19163 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:43:24.972364 19163 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.190:37457
I20260925 02:43:24.972386 19231 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.190:37457 every 8 connection(s)
I20260925 02:43:24.973332 19163 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
I20260925 02:43:24.976610 19232 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:43:24.981316 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 19163
I20260925 02:43:24.981453 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790304178325258-13942-0/minicluster-data/master-0/wal/instance
I20260925 02:43:24.981745 13942 external_mini_cluster.cc:1428] Setting key 682659a4f821bbc8730a4beb1107d3a7
I20260925 02:43:24.982688 19232 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b18187afb3104ee0a06222033ab61b9c: Bootstrap starting.
I20260925 02:43:24.985024 19232 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b18187afb3104ee0a06222033ab61b9c: Neither blocks nor log segments found. Creating new log.
I20260925 02:43:24.985873 19232 log.cc:824] T 00000000000000000000000000000000 P b18187afb3104ee0a06222033ab61b9c: Log is configured to *not* fsync() on all Append() calls
I20260925 02:43:24.988030 19232 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b18187afb3104ee0a06222033ab61b9c: No bootstrap required, opened a new log
I20260925 02:43:24.990985 19235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:24.983176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35052 (local address 127.13.157.190:37457)
0925 02:43:24.983823 (+ 647us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:24.983838 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:24.983874 (+ 36us) server_negotiation.cc:407] Connection header received
0925 02:43:24.984504 (+ 630us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:24.984523 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:24.984941 (+ 418us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:24.985253 (+ 312us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:24.985627 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:24.986624 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:24.987805 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:24.988313 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:24.988431 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:24.989224 (+ 793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:24.989229 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:24.989242 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:43:24.989631 (+ 389us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:24.989687 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:43:24.989707 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:24.989780 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:24.989923 (+ 143us) server_negotiation.cc:299] Negotiation successful
0925 02:43:24.990190 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":274,"thread_start_us":124,"threads_started":1}
I20260925 02:43:24.991999 19232 raft_consensus.cc:348] T 00000000000000000000000000000000 P b18187afb3104ee0a06222033ab61b9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b18187afb3104ee0a06222033ab61b9c" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 37457 } }
I20260925 02:43:24.992254 19232 raft_consensus.cc:374] T 00000000000000000000000000000000 P b18187afb3104ee0a06222033ab61b9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:43:24.992311 19232 raft_consensus.cc:729] T 00000000000000000000000000000000 P b18187afb3104ee0a06222033ab61b9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b18187afb3104ee0a06222033ab61b9c, State: Initialized, Role: FOLLOWER
I20260925 02:43:24.992758 19232 consensus_queue.cc:260] T 00000000000000000000000000000000 P b18187afb3104ee0a06222033ab61b9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b18187afb3104ee0a06222033ab61b9c" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 37457 } }
I20260925 02:43:24.992918 19232 raft_consensus.cc:388] T 00000000000000000000000000000000 P b18187afb3104ee0a06222033ab61b9c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 02:43:24.992982 19232 raft_consensus.cc:482] T 00000000000000000000000000000000 P b18187afb3104ee0a06222033ab61b9c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 02:43:24.993074 19232 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b18187afb3104ee0a06222033ab61b9c [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:43:24.993973 13942 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 02:43:24.994105 19232 raft_consensus.cc:504] T 00000000000000000000000000000000 P b18187afb3104ee0a06222033ab61b9c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b18187afb3104ee0a06222033ab61b9c" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 37457 } }
I20260925 02:43:24.994444 19232 leader_election.cc:302] T 00000000000000000000000000000000 P b18187afb3104ee0a06222033ab61b9c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b18187afb3104ee0a06222033ab61b9c; no voters:
I20260925 02:43:24.994745 19232 leader_election.cc:288] T 00000000000000000000000000000000 P b18187afb3104ee0a06222033ab61b9c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 02:43:24.994899 19237 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b18187afb3104ee0a06222033ab61b9c [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:43:24.995647 19237 raft_consensus.cc:686] T 00000000000000000000000000000000 P b18187afb3104ee0a06222033ab61b9c [term 1 LEADER]: Becoming Leader. State: Replica: b18187afb3104ee0a06222033ab61b9c, State: Running, Role: LEADER
I20260925 02:43:24.995985 19237 consensus_queue.cc:237] T 00000000000000000000000000000000 P b18187afb3104ee0a06222033ab61b9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b18187afb3104ee0a06222033ab61b9c" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 37457 } }
I20260925 02:43:24.996027 19232 sys_catalog.cc:564] T 00000000000000000000000000000000 P b18187afb3104ee0a06222033ab61b9c [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:43:24.997483 19239 sys_catalog.cc:455] T 00000000000000000000000000000000 P b18187afb3104ee0a06222033ab61b9c [sys.catalog]: SysCatalogTable state changed. Reason: New leader b18187afb3104ee0a06222033ab61b9c. Latest consensus state: current_term: 1 leader_uuid: "b18187afb3104ee0a06222033ab61b9c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b18187afb3104ee0a06222033ab61b9c" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 37457 } } }
I20260925 02:43:24.997532 19238 sys_catalog.cc:455] T 00000000000000000000000000000000 P b18187afb3104ee0a06222033ab61b9c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b18187afb3104ee0a06222033ab61b9c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b18187afb3104ee0a06222033ab61b9c" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 37457 } } }
I20260925 02:43:24.997612 19239 sys_catalog.cc:458] T 00000000000000000000000000000000 P b18187afb3104ee0a06222033ab61b9c [sys.catalog]: This master's current role is: LEADER
I20260925 02:43:24.997644 19238 sys_catalog.cc:458] T 00000000000000000000000000000000 P b18187afb3104ee0a06222033ab61b9c [sys.catalog]: This master's current role is: LEADER
I20260925 02:43:24.997957 19252 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:43:25.001551 19252 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:43:25.008096 19252 catalog_manager.cc:1347] Generated new cluster ID: 1c820a2d307f42ed8390da835268454b
I20260925 02:43:25.008185 19252 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 02:43:25.022039 19252 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 02:43:25.023056 19252 catalog_manager.cc:1504] Loading token signing keys...
I20260925 02:43:25.028410 19252 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b18187afb3104ee0a06222033ab61b9c: Generated new TSK 0
I20260925 02:43:25.029284 19252 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260925 02:43:25.061722 19263 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260925 02:43:25.061868 19263 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260925 02:43:25.062563 19235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:25.045614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35054 (local address 127.13.157.190:37457)
0925 02:43:25.045737 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:25.045741 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:25.045926 (+ 185us) server_negotiation.cc:407] Connection header received
0925 02:43:25.046149 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:25.046151 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:25.046201 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:25.046283 (+ 82us) server_negotiation.cc:226] Negotiated authn=JWT
0925 02:43:25.046544 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:25.047132 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:25.047992 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:25.048220 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:25.048304 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:25.048642 (+ 338us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0925 02:43:25.062039 (+ 13397us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33137/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0925 02:43:25.062406 (+ 367us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:35054: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33137/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":40}
W20260925 02:43:25.062660 19235 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:35054: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33137/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260925 02:43:25.063877 19262 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:43:25.045514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:37457 (local address 127.0.0.1:35054)
0925 02:43:25.045846 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:25.045858 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:25.045989 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:25.046311 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:25.046314 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:25.046322 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0925 02:43:25.046444 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:25.046449 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:25.047235 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:25.047238 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:25.047882 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:25.047890 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:25.048324 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:25.048328 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:25.048420 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:25.048464 (+ 44us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0925 02:43:25.062354 (+ 13890us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33137/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0925 02:43:25.062628 (+ 274us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.13.157.190:37457: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33137/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":113,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.13.157.190:37457: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33137/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260925 02:43:25.064590 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 19163
2026-09-25T02:43:25Z chronyd exiting
I20260925 02:43:25.102263 13942 test_util.cc:175] -----------------------------------------------
I20260925 02:43:25.102356 13942 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790304178325258-13942-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (330 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260925 02:43:25.103500 13942 mini_oidc.cc:152] Starting JWKS server
I20260925 02:43:25.104357 13942 webserver.cc:460] Webserver started at https://127.0.0.1:38437/ using document root <none> and password file <none>
I20260925 02:43:25.104456 13942 mini_oidc.cc:189] Starting OIDC Discovery server
I20260925 02:43:25.104717 13942 webserver.cc:460] Webserver started at http://127.0.0.1:33681/ using document root <none> and password file <none>
2026-09-25T02:43:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T02:43:25Z Disabled control of system clock
I20260925 02:43:25.113685 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790304178325258-13942-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790304178325258-13942-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790304178325258-13942-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.157.190:40791
--webserver_interface=127.13.157.190
--webserver_port=0
--builtin_ntp_servers=127.13.157.148:36185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.157.190:40791
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38437/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790304178325258-13942-0/testchainca.cert with env {}
W20260925 02:43:25.226112 19269 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:43:25.226404 19269 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:43:25.226444 19269 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:43:25.230531 19269 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:43:25.230613 19269 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:43:25.230628 19269 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:43:25.230645 19269 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:43:25.230659 19269 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260925 02:43:25.230674 19269 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260925 02:43:25.230685 19269 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:38437/jwks.json
I20260925 02:43:25.236330 19269 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:36185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790304178325258-13942-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790304178325258-13942-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.157.190:40791
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.157.190:40791
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38437/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.157.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790304178325258-13942-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19269
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790304178325258-13942-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:43:25.237416 19269 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:43:25.238497 19269 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 02:43:25.247076 19269 server_base.cc:1034] running on GCE node
W20260925 02:43:25.247040 19275 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:25.247040 19277 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:25.247098 19274 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:43:25.247946 19269 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:43:25.249122 19269 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:43:25.250300 19269 hybrid_clock.cc:648] HybridClock initialized: now 1790304205250284 us; error 28 us; skew 500 ppm
I20260925 02:43:25.252164 19269 webserver.cc:460] Webserver started at http://127.13.157.190:42733/ using document root <none> and password file <none>
I20260925 02:43:25.252728 19269 fs_manager.cc:359] Metadata directory not provided
I20260925 02:43:25.252787 19269 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:43:25.252955 19269 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:43:25.255051 19269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790304178325258-13942-0/minicluster-data/master-0/data/instance:
uuid: "ba40287d2c754b33937679e5897bd743"
format_stamp: "Formatted at 2026-09-25 02:43:25 on dist-test-slave-kznw"
server_key: "b9b3d7287c43c8810736e548dc7d45bb"
server_key_iv: "69175d469dbe6c1409c6c960038d6996"
server_key_version: "encryptionkey@0"
I20260925 02:43:25.255611 19269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790304178325258-13942-0/minicluster-data/master-0/wal/instance:
uuid: "ba40287d2c754b33937679e5897bd743"
format_stamp: "Formatted at 2026-09-25 02:43:25 on dist-test-slave-kznw"
server_key: "b9b3d7287c43c8810736e548dc7d45bb"
server_key_iv: "69175d469dbe6c1409c6c960038d6996"
server_key_version: "encryptionkey@0"
I20260925 02:43:25.259380 19269 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260925 02:43:25.261840 19283 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:25.263242 19269 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260925 02:43:25.263355 19269 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790304178325258-13942-0/minicluster-data/master-0/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790304178325258-13942-0/minicluster-data/master-0/wal
uuid: "ba40287d2c754b33937679e5897bd743"
format_stamp: "Formatted at 2026-09-25 02:43:25 on dist-test-slave-kznw"
server_key: "b9b3d7287c43c8810736e548dc7d45bb"
server_key_iv: "69175d469dbe6c1409c6c960038d6996"
server_key_version: "encryptionkey@0"
I20260925 02:43:25.263460 19269 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790304178325258-13942-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790304178325258-13942-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790304178325258-13942-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:43:25.281848 19269 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:43:25.282543 19269 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:43:25.282739 19269 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:43:25.290072 19269 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.190:40791
I20260925 02:43:25.290091 19337 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.190:40791 every 8 connection(s)
I20260925 02:43:25.291152 19269 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
I20260925 02:43:25.294545 19338 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:43:25.299664 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 19269
I20260925 02:43:25.299783 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790304178325258-13942-0/minicluster-data/master-0/wal/instance
I20260925 02:43:25.300101 13942 external_mini_cluster.cc:1428] Setting key 9399fd025669e2ab2d1ccf62f6576f91
I20260925 02:43:25.300521 19338 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba40287d2c754b33937679e5897bd743: Bootstrap starting.
I20260925 02:43:25.303081 19338 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ba40287d2c754b33937679e5897bd743: Neither blocks nor log segments found. Creating new log.
I20260925 02:43:25.303876 19338 log.cc:824] T 00000000000000000000000000000000 P ba40287d2c754b33937679e5897bd743: Log is configured to *not* fsync() on all Append() calls
I20260925 02:43:25.306123 19338 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba40287d2c754b33937679e5897bd743: No bootstrap required, opened a new log
I20260925 02:43:25.309746 19341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:25.301582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41112 (local address 127.13.157.190:40791)
0925 02:43:25.302225 (+ 643us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:25.302240 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:25.302277 (+ 37us) server_negotiation.cc:407] Connection header received
0925 02:43:25.303038 (+ 761us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:25.303059 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:25.303503 (+ 444us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:25.303788 (+ 285us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:25.304195 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:25.305154 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:25.306360 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:25.306852 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:25.307031 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:25.307864 (+ 833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:25.307875 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:25.307888 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:43:25.308362 (+ 474us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:25.308418 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:43:25.308439 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:25.308513 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:25.308662 (+ 149us) server_negotiation.cc:299] Negotiation successful
0925 02:43:25.308929 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":271,"thread_start_us":121,"threads_started":1}
I20260925 02:43:25.310256 19338 raft_consensus.cc:348] T 00000000000000000000000000000000 P ba40287d2c754b33937679e5897bd743 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba40287d2c754b33937679e5897bd743" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 40791 } }
I20260925 02:43:25.310482 19338 raft_consensus.cc:374] T 00000000000000000000000000000000 P ba40287d2c754b33937679e5897bd743 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:43:25.310518 19338 raft_consensus.cc:729] T 00000000000000000000000000000000 P ba40287d2c754b33937679e5897bd743 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba40287d2c754b33937679e5897bd743, State: Initialized, Role: FOLLOWER
I20260925 02:43:25.311017 19338 consensus_queue.cc:260] T 00000000000000000000000000000000 P ba40287d2c754b33937679e5897bd743 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba40287d2c754b33937679e5897bd743" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 40791 } }
I20260925 02:43:25.311183 19338 raft_consensus.cc:388] T 00000000000000000000000000000000 P ba40287d2c754b33937679e5897bd743 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 02:43:25.311245 19338 raft_consensus.cc:482] T 00000000000000000000000000000000 P ba40287d2c754b33937679e5897bd743 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 02:43:25.311348 19338 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ba40287d2c754b33937679e5897bd743 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:43:25.312433 19338 raft_consensus.cc:504] T 00000000000000000000000000000000 P ba40287d2c754b33937679e5897bd743 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba40287d2c754b33937679e5897bd743" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 40791 } }
I20260925 02:43:25.312752 13942 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 02:43:25.312801 19338 leader_election.cc:302] T 00000000000000000000000000000000 P ba40287d2c754b33937679e5897bd743 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ba40287d2c754b33937679e5897bd743; no voters:
I20260925 02:43:25.313129 19338 leader_election.cc:288] T 00000000000000000000000000000000 P ba40287d2c754b33937679e5897bd743 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 02:43:25.313225 19343 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ba40287d2c754b33937679e5897bd743 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:43:25.313441 19343 raft_consensus.cc:686] T 00000000000000000000000000000000 P ba40287d2c754b33937679e5897bd743 [term 1 LEADER]: Becoming Leader. State: Replica: ba40287d2c754b33937679e5897bd743, State: Running, Role: LEADER
I20260925 02:43:25.313783 19343 consensus_queue.cc:237] T 00000000000000000000000000000000 P ba40287d2c754b33937679e5897bd743 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba40287d2c754b33937679e5897bd743" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 40791 } }
I20260925 02:43:25.314479 19338 sys_catalog.cc:564] T 00000000000000000000000000000000 P ba40287d2c754b33937679e5897bd743 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:43:25.315475 19345 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba40287d2c754b33937679e5897bd743 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ba40287d2c754b33937679e5897bd743. Latest consensus state: current_term: 1 leader_uuid: "ba40287d2c754b33937679e5897bd743" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba40287d2c754b33937679e5897bd743" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 40791 } } }
I20260925 02:43:25.315522 19344 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba40287d2c754b33937679e5897bd743 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ba40287d2c754b33937679e5897bd743" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba40287d2c754b33937679e5897bd743" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 40791 } } }
I20260925 02:43:25.315609 19345 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba40287d2c754b33937679e5897bd743 [sys.catalog]: This master's current role is: LEADER
I20260925 02:43:25.315618 19344 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba40287d2c754b33937679e5897bd743 [sys.catalog]: This master's current role is: LEADER
I20260925 02:43:25.316022 19352 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:43:25.319145 19352 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:43:25.325095 19352 catalog_manager.cc:1347] Generated new cluster ID: ba3a9db81b6b475090e87ba8e41dc643
I20260925 02:43:25.325168 19352 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 02:43:25.366943 19352 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 02:43:25.368309 19352 catalog_manager.cc:1504] Loading token signing keys...
I20260925 02:43:25.372808 19352 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ba40287d2c754b33937679e5897bd743: Generated new TSK 0
I20260925 02:43:25.373507 19352 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260925 02:43:25.402621 19368 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260925 02:43:25.402732 19368 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260925 02:43:25.403530 19341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:25.387227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41116 (local address 127.13.157.190:40791)
0925 02:43:25.387371 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:25.387375 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:25.387506 (+ 131us) server_negotiation.cc:407] Connection header received
0925 02:43:25.387682 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:25.387685 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:25.387746 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:25.387862 (+ 116us) server_negotiation.cc:226] Negotiated authn=JWT
0925 02:43:25.388138 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:25.388958 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:25.389870 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:25.390229 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:25.390328 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:25.390631 (+ 303us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0925 02:43:25.403077 (+ 12446us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38437/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0925 02:43:25.403383 (+ 306us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:41116: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38437/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":57}
W20260925 02:43:25.403606 19341 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:41116: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38437/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260925 02:43:25.403688 19367 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:43:25.387138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:40791 (local address 127.0.0.1:41116)
0925 02:43:25.387427 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:25.387439 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:25.387550 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:25.387896 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:25.387899 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:25.387906 (+ 7us) client_negotiation.cc:189] Negotiated authn=JWT
0925 02:43:25.388016 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:25.388021 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:25.389118 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:25.389121 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:25.389761 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:25.389770 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:25.390357 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:25.390361 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:25.390427 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:25.390451 (+ 24us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0925 02:43:25.403335 (+ 12884us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38437/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0925 02:43:25.403499 (+ 164us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.13.157.190:40791: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38437/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":97,"threads_started":1}
I20260925 02:43:25.404454 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 19269
2026-09-25T02:43:25Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (340 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260925 02:43:25.443603 13942 mini_oidc.cc:152] Starting JWKS server
I20260925 02:43:25.444450 13942 webserver.cc:460] Webserver started at https://127.0.0.1:39325/ using document root <none> and password file <none>
I20260925 02:43:25.444540 13942 mini_oidc.cc:189] Starting OIDC Discovery server
I20260925 02:43:25.444801 13942 webserver.cc:460] Webserver started at http://127.0.0.1:41677/ using document root <none> and password file <none>
2026-09-25T02:43:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T02:43:25Z Disabled control of system clock
I20260925 02:43:25.453722 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790304178325258-13942-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790304178325258-13942-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790304178325258-13942-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.157.190:36215
--webserver_interface=127.13.157.190
--webserver_port=0
--builtin_ntp_servers=127.13.157.148:40905
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.157.190:36215
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39325/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260925 02:43:25.563486 19375 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:43:25.563828 19375 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:43:25.563890 19375 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:43:25.567883 19375 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:43:25.567939 19375 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:43:25.567953 19375 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:43:25.567968 19375 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:43:25.567982 19375 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260925 02:43:25.567996 19375 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260925 02:43:25.568007 19375 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:39325/jwks.json
I20260925 02:43:25.572742 19375 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:40905
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790304178325258-13942-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790304178325258-13942-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.157.190:36215
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.157.190:36215
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39325/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.157.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19375
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790304178325258-13942-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:43:25.573771 19375 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:43:25.575256 19375 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:43:25.583493 19380 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:25.583597 19381 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:25.583500 19383 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:43:25.583578 19375 server_base.cc:1034] running on GCE node
I20260925 02:43:25.584239 19375 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:43:25.585309 19375 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:43:25.586480 19375 hybrid_clock.cc:648] HybridClock initialized: now 1790304205586466 us; error 28 us; skew 500 ppm
I20260925 02:43:25.588500 19375 webserver.cc:460] Webserver started at http://127.13.157.190:34643/ using document root <none> and password file <none>
I20260925 02:43:25.589082 19375 fs_manager.cc:359] Metadata directory not provided
I20260925 02:43:25.589145 19375 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:43:25.589354 19375 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:43:25.591452 19375 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790304178325258-13942-0/minicluster-data/master-0/data/instance:
uuid: "cbbe874b70f5457a82d4be28ce984807"
format_stamp: "Formatted at 2026-09-25 02:43:25 on dist-test-slave-kznw"
server_key: "845be40564e3000d1e64b3126a283928"
server_key_iv: "72a93a88812578c9f3e0d75c5bc3303e"
server_key_version: "encryptionkey@0"
I20260925 02:43:25.592006 19375 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790304178325258-13942-0/minicluster-data/master-0/wal/instance:
uuid: "cbbe874b70f5457a82d4be28ce984807"
format_stamp: "Formatted at 2026-09-25 02:43:25 on dist-test-slave-kznw"
server_key: "845be40564e3000d1e64b3126a283928"
server_key_iv: "72a93a88812578c9f3e0d75c5bc3303e"
server_key_version: "encryptionkey@0"
I20260925 02:43:25.595773 19375 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260925 02:43:25.598322 19389 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:25.599756 19375 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260925 02:43:25.599884 19375 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790304178325258-13942-0/minicluster-data/master-0/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790304178325258-13942-0/minicluster-data/master-0/wal
uuid: "cbbe874b70f5457a82d4be28ce984807"
format_stamp: "Formatted at 2026-09-25 02:43:25 on dist-test-slave-kznw"
server_key: "845be40564e3000d1e64b3126a283928"
server_key_iv: "72a93a88812578c9f3e0d75c5bc3303e"
server_key_version: "encryptionkey@0"
I20260925 02:43:25.600000 19375 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790304178325258-13942-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790304178325258-13942-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790304178325258-13942-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:43:25.623931 19375 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:43:25.624732 19375 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:43:25.624929 19375 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:43:25.632498 19375 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.190:36215
I20260925 02:43:25.632508 19442 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.190:36215 every 8 connection(s)
I20260925 02:43:25.633505 19375 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
I20260925 02:43:25.637792 19443 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:43:25.639688 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 19375
I20260925 02:43:25.639828 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790304178325258-13942-0/minicluster-data/master-0/wal/instance
I20260925 02:43:25.640095 13942 external_mini_cluster.cc:1428] Setting key ae71ce2f4ec92a27344e993840021302
I20260925 02:43:25.643884 19443 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cbbe874b70f5457a82d4be28ce984807: Bootstrap starting.
I20260925 02:43:25.646189 19443 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cbbe874b70f5457a82d4be28ce984807: Neither blocks nor log segments found. Creating new log.
I20260925 02:43:25.646942 19443 log.cc:824] T 00000000000000000000000000000000 P cbbe874b70f5457a82d4be28ce984807: Log is configured to *not* fsync() on all Append() calls
I20260925 02:43:25.649091 19443 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cbbe874b70f5457a82d4be28ce984807: No bootstrap required, opened a new log
I20260925 02:43:25.649569 19446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:25.641460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41868 (local address 127.13.157.190:36215)
0925 02:43:25.642144 (+ 684us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:25.642160 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:25.642203 (+ 43us) server_negotiation.cc:407] Connection header received
0925 02:43:25.643002 (+ 799us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:25.643024 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:25.643439 (+ 415us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:25.643756 (+ 317us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:25.644176 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:25.645232 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:25.646112 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:25.646617 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:25.646764 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:25.647662 (+ 898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:25.647667 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:25.647680 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:43:25.648056 (+ 376us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:25.648124 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:43:25.648152 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:25.648285 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:25.648421 (+ 136us) server_negotiation.cc:299] Negotiation successful
0925 02:43:25.648690 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":310,"thread_start_us":138,"threads_started":1}
I20260925 02:43:25.652699 13942 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 02:43:25.653252 19443 raft_consensus.cc:348] T 00000000000000000000000000000000 P cbbe874b70f5457a82d4be28ce984807 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbbe874b70f5457a82d4be28ce984807" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 36215 } }
I20260925 02:43:25.653484 19443 raft_consensus.cc:374] T 00000000000000000000000000000000 P cbbe874b70f5457a82d4be28ce984807 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:43:25.653537 19443 raft_consensus.cc:729] T 00000000000000000000000000000000 P cbbe874b70f5457a82d4be28ce984807 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cbbe874b70f5457a82d4be28ce984807, State: Initialized, Role: FOLLOWER
I20260925 02:43:25.653977 19443 consensus_queue.cc:260] T 00000000000000000000000000000000 P cbbe874b70f5457a82d4be28ce984807 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbbe874b70f5457a82d4be28ce984807" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 36215 } }
I20260925 02:43:25.654109 19443 raft_consensus.cc:388] T 00000000000000000000000000000000 P cbbe874b70f5457a82d4be28ce984807 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 02:43:25.654170 19443 raft_consensus.cc:482] T 00000000000000000000000000000000 P cbbe874b70f5457a82d4be28ce984807 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 02:43:25.654260 19443 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cbbe874b70f5457a82d4be28ce984807 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:43:25.655282 19443 raft_consensus.cc:504] T 00000000000000000000000000000000 P cbbe874b70f5457a82d4be28ce984807 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbbe874b70f5457a82d4be28ce984807" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 36215 } }
I20260925 02:43:25.655597 19443 leader_election.cc:302] T 00000000000000000000000000000000 P cbbe874b70f5457a82d4be28ce984807 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cbbe874b70f5457a82d4be28ce984807; no voters:
I20260925 02:43:25.655885 19443 leader_election.cc:288] T 00000000000000000000000000000000 P cbbe874b70f5457a82d4be28ce984807 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 02:43:25.655980 19448 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cbbe874b70f5457a82d4be28ce984807 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:43:25.656206 19448 raft_consensus.cc:686] T 00000000000000000000000000000000 P cbbe874b70f5457a82d4be28ce984807 [term 1 LEADER]: Becoming Leader. State: Replica: cbbe874b70f5457a82d4be28ce984807, State: Running, Role: LEADER
I20260925 02:43:25.656504 19448 consensus_queue.cc:237] T 00000000000000000000000000000000 P cbbe874b70f5457a82d4be28ce984807 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbbe874b70f5457a82d4be28ce984807" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 36215 } }
I20260925 02:43:25.657191 19443 sys_catalog.cc:564] T 00000000000000000000000000000000 P cbbe874b70f5457a82d4be28ce984807 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:43:25.658249 19450 sys_catalog.cc:455] T 00000000000000000000000000000000 P cbbe874b70f5457a82d4be28ce984807 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cbbe874b70f5457a82d4be28ce984807. Latest consensus state: current_term: 1 leader_uuid: "cbbe874b70f5457a82d4be28ce984807" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbbe874b70f5457a82d4be28ce984807" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 36215 } } }
I20260925 02:43:25.658380 19450 sys_catalog.cc:458] T 00000000000000000000000000000000 P cbbe874b70f5457a82d4be28ce984807 [sys.catalog]: This master's current role is: LEADER
I20260925 02:43:25.658639 19449 sys_catalog.cc:455] T 00000000000000000000000000000000 P cbbe874b70f5457a82d4be28ce984807 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cbbe874b70f5457a82d4be28ce984807" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbbe874b70f5457a82d4be28ce984807" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 36215 } } }
I20260925 02:43:25.658748 19449 sys_catalog.cc:458] T 00000000000000000000000000000000 P cbbe874b70f5457a82d4be28ce984807 [sys.catalog]: This master's current role is: LEADER
I20260925 02:43:25.658833 19457 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:43:25.661970 19457 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:43:25.667680 19457 catalog_manager.cc:1347] Generated new cluster ID: f873a221af9c4e5aa3325234c8041656
I20260925 02:43:25.667735 19457 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 02:43:25.678678 19457 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 02:43:25.679597 19457 catalog_manager.cc:1504] Loading token signing keys...
I20260925 02:43:25.689514 19457 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cbbe874b70f5457a82d4be28ce984807: Generated new TSK 0
I20260925 02:43:25.690225 19457 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 02:43:25.704591 19446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:25.699490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41878 (local address 127.13.157.190:36215)
0925 02:43:25.699618 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:25.699622 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:25.699808 (+ 186us) server_negotiation.cc:407] Connection header received
0925 02:43:25.699969 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:25.699972 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:25.700024 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:25.700156 (+ 132us) server_negotiation.cc:226] Negotiated authn=JWT
0925 02:43:25.700382 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:25.700912 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:25.702146 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:25.702363 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:25.702440 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:25.702747 (+ 307us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0925 02:43:25.704220 (+ 1473us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39325/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0925 02:43:25.704442 (+ 222us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:41878: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39325/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":44}
W20260925 02:43:25.704663 19446 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:41878: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39325/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260925 02:43:25.706449 19473 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:43:25.699398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:36215 (local address 127.0.0.1:41878)
0925 02:43:25.699705 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:25.699733 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:25.699854 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:25.700170 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:25.700173 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:25.700182 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0925 02:43:25.700294 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:25.700300 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:25.701017 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:25.701021 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:25.702026 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:25.702036 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:25.702471 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:25.702475 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:25.702572 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:25.702602 (+ 30us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0925 02:43:25.704398 (+ 1796us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39325/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0925 02:43:25.706266 (+ 1868us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.13.157.190:36215: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39325/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":220,"mutex_wait_us":1664,"thread_start_us":86,"threads_started":1}
I20260925 02:43:25.707094 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 19375
E20260925 02:43:25.715692 19474 webserver.cc:533] Webserver: sslize failed:
E20260925 02:43:25.715766 19474 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-09-25T02:43:25Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (302 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-09-25T02:43:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T02:43:25Z Disabled control of system clock
I20260925 02:43:25.754729 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790304178325258-13942-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790304178325258-13942-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790304178325258-13942-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.157.190:36009
--webserver_interface=127.13.157.190
--webserver_port=0
--builtin_ntp_servers=127.13.157.148:34057
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.157.190:36009
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790304178325258-13942-0/testchainca.cert with env {}
W20260925 02:43:25.863243 19478 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:43:25.863534 19478 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:43:25.863591 19478 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:43:25.867679 19478 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:43:25.867753 19478 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:43:25.867774 19478 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:43:25.867794 19478 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:43:25.867812 19478 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260925 02:43:25.867835 19478 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260925 02:43:25.867862 19478 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260925 02:43:25.873440 19478 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:34057
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790304178325258-13942-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790304178325258-13942-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.157.190:36009
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.157.190:36009
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.157.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790304178325258-13942-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19478
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790304178325258-13942-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:43:25.874575 19478 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:43:25.875628 19478 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:43:25.883870 19484 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:25.883881 19485 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:25.883889 19487 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:43:25.884042 19478 server_base.cc:1034] running on GCE node
I20260925 02:43:25.884681 19478 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:43:25.885803 19478 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:43:25.886993 19478 hybrid_clock.cc:648] HybridClock initialized: now 1790304205886967 us; error 36 us; skew 500 ppm
I20260925 02:43:25.889107 19478 webserver.cc:460] Webserver started at http://127.13.157.190:34095/ using document root <none> and password file <none>
I20260925 02:43:25.889746 19478 fs_manager.cc:359] Metadata directory not provided
I20260925 02:43:25.889820 19478 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:43:25.890033 19478 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:43:25.892171 19478 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790304178325258-13942-0/minicluster-data/master-0/data/instance:
uuid: "a4c7ffe9862b4be2b067bd2141c6a250"
format_stamp: "Formatted at 2026-09-25 02:43:25 on dist-test-slave-kznw"
server_key: "1069c1ed8b04f937233022cb7eac9006"
server_key_iv: "6a0f969ba9baea5459d5f516fb4ff21c"
server_key_version: "encryptionkey@0"
I20260925 02:43:25.892743 19478 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790304178325258-13942-0/minicluster-data/master-0/wal/instance:
uuid: "a4c7ffe9862b4be2b067bd2141c6a250"
format_stamp: "Formatted at 2026-09-25 02:43:25 on dist-test-slave-kznw"
server_key: "1069c1ed8b04f937233022cb7eac9006"
server_key_iv: "6a0f969ba9baea5459d5f516fb4ff21c"
server_key_version: "encryptionkey@0"
I20260925 02:43:25.896639 19478 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260925 02:43:25.899413 19493 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:25.900784 19478 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 02:43:25.900913 19478 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790304178325258-13942-0/minicluster-data/master-0/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790304178325258-13942-0/minicluster-data/master-0/wal
uuid: "a4c7ffe9862b4be2b067bd2141c6a250"
format_stamp: "Formatted at 2026-09-25 02:43:25 on dist-test-slave-kznw"
server_key: "1069c1ed8b04f937233022cb7eac9006"
server_key_iv: "6a0f969ba9baea5459d5f516fb4ff21c"
server_key_version: "encryptionkey@0"
I20260925 02:43:25.901026 19478 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790304178325258-13942-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790304178325258-13942-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790304178325258-13942-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:43:25.929692 19478 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:43:25.930501 19478 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:43:25.930722 19478 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:43:25.937450 19478 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.190:36009
I20260925 02:43:25.937472 19547 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.190:36009 every 8 connection(s)
I20260925 02:43:25.938390 19478 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
I20260925 02:43:25.940361 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 19478
I20260925 02:43:25.940461 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790304178325258-13942-0/minicluster-data/master-0/wal/instance
I20260925 02:43:25.940739 13942 external_mini_cluster.cc:1428] Setting key 3a43ebc7a12ed31d091a08e15486ba2c
I20260925 02:43:25.941519 19548 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:43:25.948704 19548 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4c7ffe9862b4be2b067bd2141c6a250: Bootstrap starting.
I20260925 02:43:25.951270 19551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:25.942155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45994 (local address 127.13.157.190:36009)
0925 02:43:25.942840 (+ 685us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:25.942860 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:25.942929 (+ 69us) server_negotiation.cc:407] Connection header received
0925 02:43:25.943719 (+ 790us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:25.943744 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:25.944255 (+ 511us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:25.944652 (+ 397us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:25.945112 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:25.946556 (+ 1444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:25.947540 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:25.948226 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:25.948385 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:25.949195 (+ 810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:25.949201 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:25.949216 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:43:25.949712 (+ 496us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:25.949780 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:43:25.949807 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:25.949910 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:25.950002 (+ 92us) server_negotiation.cc:299] Negotiation successful
0925 02:43:25.950288 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":281,"thread_start_us":107,"threads_started":1}
I20260925 02:43:25.951514 19548 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a4c7ffe9862b4be2b067bd2141c6a250: Neither blocks nor log segments found. Creating new log.
I20260925 02:43:25.952380 19548 log.cc:824] T 00000000000000000000000000000000 P a4c7ffe9862b4be2b067bd2141c6a250: Log is configured to *not* fsync() on all Append() calls
I20260925 02:43:25.954255 13942 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 02:43:25.954628 19548 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4c7ffe9862b4be2b067bd2141c6a250: No bootstrap required, opened a new log
I20260925 02:43:25.958310 19548 raft_consensus.cc:348] T 00000000000000000000000000000000 P a4c7ffe9862b4be2b067bd2141c6a250 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4c7ffe9862b4be2b067bd2141c6a250" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 36009 } }
I20260925 02:43:25.958560 19548 raft_consensus.cc:374] T 00000000000000000000000000000000 P a4c7ffe9862b4be2b067bd2141c6a250 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:43:25.958628 19548 raft_consensus.cc:729] T 00000000000000000000000000000000 P a4c7ffe9862b4be2b067bd2141c6a250 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4c7ffe9862b4be2b067bd2141c6a250, State: Initialized, Role: FOLLOWER
I20260925 02:43:25.959067 19548 consensus_queue.cc:260] T 00000000000000000000000000000000 P a4c7ffe9862b4be2b067bd2141c6a250 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4c7ffe9862b4be2b067bd2141c6a250" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 36009 } }
I20260925 02:43:25.959213 19548 raft_consensus.cc:388] T 00000000000000000000000000000000 P a4c7ffe9862b4be2b067bd2141c6a250 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 02:43:25.959275 19548 raft_consensus.cc:482] T 00000000000000000000000000000000 P a4c7ffe9862b4be2b067bd2141c6a250 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 02:43:25.959372 19548 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a4c7ffe9862b4be2b067bd2141c6a250 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:43:25.960330 19548 raft_consensus.cc:504] T 00000000000000000000000000000000 P a4c7ffe9862b4be2b067bd2141c6a250 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4c7ffe9862b4be2b067bd2141c6a250" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 36009 } }
I20260925 02:43:25.960657 19548 leader_election.cc:302] T 00000000000000000000000000000000 P a4c7ffe9862b4be2b067bd2141c6a250 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a4c7ffe9862b4be2b067bd2141c6a250; no voters:
I20260925 02:43:25.960954 19548 leader_election.cc:288] T 00000000000000000000000000000000 P a4c7ffe9862b4be2b067bd2141c6a250 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 02:43:25.961108 19553 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a4c7ffe9862b4be2b067bd2141c6a250 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:43:25.961323 19553 raft_consensus.cc:686] T 00000000000000000000000000000000 P a4c7ffe9862b4be2b067bd2141c6a250 [term 1 LEADER]: Becoming Leader. State: Replica: a4c7ffe9862b4be2b067bd2141c6a250, State: Running, Role: LEADER
I20260925 02:43:25.961632 19553 consensus_queue.cc:237] T 00000000000000000000000000000000 P a4c7ffe9862b4be2b067bd2141c6a250 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4c7ffe9862b4be2b067bd2141c6a250" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 36009 } }
I20260925 02:43:25.962283 19548 sys_catalog.cc:564] T 00000000000000000000000000000000 P a4c7ffe9862b4be2b067bd2141c6a250 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:43:25.962751 19553 sys_catalog.cc:455] T 00000000000000000000000000000000 P a4c7ffe9862b4be2b067bd2141c6a250 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a4c7ffe9862b4be2b067bd2141c6a250. Latest consensus state: current_term: 1 leader_uuid: "a4c7ffe9862b4be2b067bd2141c6a250" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4c7ffe9862b4be2b067bd2141c6a250" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 36009 } } }
I20260925 02:43:25.962867 19553 sys_catalog.cc:458] T 00000000000000000000000000000000 P a4c7ffe9862b4be2b067bd2141c6a250 [sys.catalog]: This master's current role is: LEADER
I20260925 02:43:25.963318 19555 sys_catalog.cc:455] T 00000000000000000000000000000000 P a4c7ffe9862b4be2b067bd2141c6a250 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a4c7ffe9862b4be2b067bd2141c6a250" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4c7ffe9862b4be2b067bd2141c6a250" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 36009 } } }
I20260925 02:43:25.963445 19558 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:43:25.963459 19555 sys_catalog.cc:458] T 00000000000000000000000000000000 P a4c7ffe9862b4be2b067bd2141c6a250 [sys.catalog]: This master's current role is: LEADER
I20260925 02:43:25.966634 19558 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:43:25.972750 19558 catalog_manager.cc:1347] Generated new cluster ID: f41cc8b9aadc47b8a0e03d22fb482cc3
I20260925 02:43:25.972806 19558 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 02:43:25.984826 19558 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 02:43:25.985729 19558 catalog_manager.cc:1504] Loading token signing keys...
I20260925 02:43:25.991184 19558 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a4c7ffe9862b4be2b067bd2141c6a250: Generated new TSK 0
I20260925 02:43:25.991833 19558 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 02:43:26.845623 19551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:26.007636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46004 (local address 127.13.157.190:36009)
0925 02:43:26.007798 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:26.007803 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:26.008045 (+ 242us) server_negotiation.cc:407] Connection header received
0925 02:43:26.008243 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:26.008247 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:26.008339 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:26.008517 (+ 178us) server_negotiation.cc:226] Negotiated authn=JWT
0925 02:43:26.008829 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:26.009653 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:26.011322 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:26.011697 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:26.011795 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:26.012306 (+ 511us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0925 02:43:26.845156 (+832850us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0925 02:43:26.845469 (+ 313us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:46004: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":73}
W20260925 02:43:26.845708 19551 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:46004: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260925 02:43:26.845890 19578 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:43:26.007435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:36009 (local address 127.0.0.1:46004)
0925 02:43:26.007938 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:26.007951 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:26.008070 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:26.008524 (+ 454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:26.008527 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:26.008535 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0925 02:43:26.008664 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:26.008670 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:26.009809 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:26.009813 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:26.011192 (+ 1379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:26.011206 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:26.011906 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:26.011910 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:26.012006 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:26.012035 (+ 29us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0925 02:43:26.845479 (+833444us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0925 02:43:26.845690 (+ 211us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.13.157.190:36009: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":103,"threads_started":1}
I20260925 02:43:26.846418 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 19478
2026-09-25T02:43:26Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (1118 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (126 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (124 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790304178325258-13942-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 02:43:27 dist-test-slave-kznw krb5kdc[19590](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 02:43:27 dist-test-slave-kznw krb5kdc[19590](info): set up 2 sockets
Sep 25 02:43:27 dist-test-slave-kznw krb5kdc[19590](info): commencing operation
krb5kdc: starting...
W20260925 02:43:28.058702 13942 mini_kdc.cc:121] Time spent starting KDC: real 0.943s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790304178325258-13942-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790304178325258-13942-0/krb5kdc/test-user.keytab.
Sep 25 02:43:28 dist-test-slave-kznw krb5kdc[19590](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304208, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T02:43:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T02:43:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.157.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.157.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.157.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab.
Entry for principal kudu/127.13.157.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab.
I20260925 02:43:28.177762 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790304178325258-13942-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790304178325258-13942-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790304178325258-13942-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.157.190:40499
--webserver_interface=127.13.157.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab
--principal=kudu/127.13.157.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:42549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.157.190:40499
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:43:28.299947 19603 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:43:28.300222 19603 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:43:28.300263 19603 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:43:28.304318 19603 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:43:28.304381 19603 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:43:28.304401 19603 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:43:28.304417 19603 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:43:28.304431 19603 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:43:28.309953 19603 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:42549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790304178325258-13942-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790304178325258-13942-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.157.190:40499
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab
--principal=kudu/127.13.157.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.157.190:40499
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.157.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19603
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790304178325258-13942-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:43:28.311054 19603 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:43:28.312050 19603 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:43:28.320623 19608 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:28.320619 19611 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:28.320648 19609 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:43:28.320833 19603 server_base.cc:1034] running on GCE node
I20260925 02:43:28.321468 19603 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:43:28.322568 19603 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:43:28.323726 19603 hybrid_clock.cc:648] HybridClock initialized: now 1790304208323691 us; error 51 us; skew 500 ppm
Sep 25 02:43:28 dist-test-slave-kznw krb5kdc[19590](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304208, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:43:28.326411 19603 init.cc:377] Logged in from keytab as kudu/127.13.157.190@KRBTEST.COM (short username kudu)
I20260925 02:43:28.327320 19603 webserver.cc:460] Webserver started at http://127.13.157.190:46661/ using document root <none> and password file <none>
I20260925 02:43:28.327843 19603 fs_manager.cc:359] Metadata directory not provided
I20260925 02:43:28.327910 19603 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:43:28.328131 19603 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:43:28.330147 19603 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790304178325258-13942-0/minicluster-data/master-0/data/instance:
uuid: "68e7a5c819f84c99a0f82b33e3f7eab0"
format_stamp: "Formatted at 2026-09-25 02:43:28 on dist-test-slave-kznw"
server_key: "e265d86b3e204b3a9d62613e4b9ce79a"
server_key_iv: "a4527f3eb77331ffbc119eb58e51ee85"
server_key_version: "encryptionkey@0"
I20260925 02:43:28.330679 19603 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790304178325258-13942-0/minicluster-data/master-0/wal/instance:
uuid: "68e7a5c819f84c99a0f82b33e3f7eab0"
format_stamp: "Formatted at 2026-09-25 02:43:28 on dist-test-slave-kznw"
server_key: "e265d86b3e204b3a9d62613e4b9ce79a"
server_key_iv: "a4527f3eb77331ffbc119eb58e51ee85"
server_key_version: "encryptionkey@0"
I20260925 02:43:28.334493 19603 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260925 02:43:28.337093 19618 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:28.338454 19603 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260925 02:43:28.338572 19603 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790304178325258-13942-0/minicluster-data/master-0/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790304178325258-13942-0/minicluster-data/master-0/wal
uuid: "68e7a5c819f84c99a0f82b33e3f7eab0"
format_stamp: "Formatted at 2026-09-25 02:43:28 on dist-test-slave-kznw"
server_key: "e265d86b3e204b3a9d62613e4b9ce79a"
server_key_iv: "a4527f3eb77331ffbc119eb58e51ee85"
server_key_version: "encryptionkey@0"
I20260925 02:43:28.338673 19603 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790304178325258-13942-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790304178325258-13942-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790304178325258-13942-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:43:28.351029 19603 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:43:28.354192 19603 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:43:28.354367 19603 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:43:28.361056 19603 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.190:40499
I20260925 02:43:28.361114 19671 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.190:40499 every 8 connection(s)
I20260925 02:43:28.361985 19603 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
I20260925 02:43:28.363826 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 19603
I20260925 02:43:28.363946 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790304178325258-13942-0/minicluster-data/master-0/wal/instance
I20260925 02:43:28.364245 13942 external_mini_cluster.cc:1428] Setting key c84ff241140a6110b7484b1461b6cdb0
I20260925 02:43:28.365237 19672 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:43:28.372299 19672 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68e7a5c819f84c99a0f82b33e3f7eab0: Bootstrap starting.
Sep 25 02:43:28 dist-test-slave-kznw krb5kdc[19590](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304208, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.157.190@KRBTEST.COM
I20260925 02:43:28.375067 19672 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 68e7a5c819f84c99a0f82b33e3f7eab0: Neither blocks nor log segments found. Creating new log.
I20260925 02:43:28.375931 19672 log.cc:824] T 00000000000000000000000000000000 P 68e7a5c819f84c99a0f82b33e3f7eab0: Log is configured to *not* fsync() on all Append() calls
I20260925 02:43:28.378350 19672 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68e7a5c819f84c99a0f82b33e3f7eab0: No bootstrap required, opened a new log
I20260925 02:43:28.379443 19675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:28.365668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59238 (local address 127.13.157.190:40499)
0925 02:43:28.366098 (+ 430us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:28.366120 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:28.366166 (+ 46us) server_negotiation.cc:407] Connection header received
0925 02:43:28.367119 (+ 953us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:28.367144 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:28.367576 (+ 432us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:28.367960 (+ 384us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:28.368666 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:28.370142 (+ 1476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:28.371043 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:28.371753 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:28.371927 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:28.374787 (+ 2860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:28.374793 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:28.374806 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:43:28.374835 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:28.376635 (+ 1800us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:28.377088 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:28.377092 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:28.377097 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:28.377208 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:28.377482 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:28.377486 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:28.377508 (+ 22us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:28.377899 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:28.378080 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:28.378360 (+ 280us) server_negotiation.cc:299] Negotiation successful
0925 02:43:28.378650 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":280,"thread_start_us":109,"threads_started":1}
I20260925 02:43:28.382452 19672 raft_consensus.cc:348] T 00000000000000000000000000000000 P 68e7a5c819f84c99a0f82b33e3f7eab0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68e7a5c819f84c99a0f82b33e3f7eab0" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 40499 } }
I20260925 02:43:28.382736 19672 raft_consensus.cc:374] T 00000000000000000000000000000000 P 68e7a5c819f84c99a0f82b33e3f7eab0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:43:28.382805 19672 raft_consensus.cc:729] T 00000000000000000000000000000000 P 68e7a5c819f84c99a0f82b33e3f7eab0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68e7a5c819f84c99a0f82b33e3f7eab0, State: Initialized, Role: FOLLOWER
I20260925 02:43:28.383311 19672 consensus_queue.cc:260] T 00000000000000000000000000000000 P 68e7a5c819f84c99a0f82b33e3f7eab0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68e7a5c819f84c99a0f82b33e3f7eab0" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 40499 } }
I20260925 02:43:28.383460 19672 raft_consensus.cc:388] T 00000000000000000000000000000000 P 68e7a5c819f84c99a0f82b33e3f7eab0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 02:43:28.383531 19672 raft_consensus.cc:482] T 00000000000000000000000000000000 P 68e7a5c819f84c99a0f82b33e3f7eab0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 02:43:28.383641 19672 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 68e7a5c819f84c99a0f82b33e3f7eab0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:43:28.384618 19672 raft_consensus.cc:504] T 00000000000000000000000000000000 P 68e7a5c819f84c99a0f82b33e3f7eab0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68e7a5c819f84c99a0f82b33e3f7eab0" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 40499 } }
I20260925 02:43:28.384967 19672 leader_election.cc:302] T 00000000000000000000000000000000 P 68e7a5c819f84c99a0f82b33e3f7eab0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 68e7a5c819f84c99a0f82b33e3f7eab0; no voters:
I20260925 02:43:28.385262 19672 leader_election.cc:288] T 00000000000000000000000000000000 P 68e7a5c819f84c99a0f82b33e3f7eab0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 02:43:28.385375 19677 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 68e7a5c819f84c99a0f82b33e3f7eab0 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:43:28.385593 19677 raft_consensus.cc:686] T 00000000000000000000000000000000 P 68e7a5c819f84c99a0f82b33e3f7eab0 [term 1 LEADER]: Becoming Leader. State: Replica: 68e7a5c819f84c99a0f82b33e3f7eab0, State: Running, Role: LEADER
I20260925 02:43:28.385900 19677 consensus_queue.cc:237] T 00000000000000000000000000000000 P 68e7a5c819f84c99a0f82b33e3f7eab0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68e7a5c819f84c99a0f82b33e3f7eab0" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 40499 } }
I20260925 02:43:28.386626 19672 sys_catalog.cc:564] T 00000000000000000000000000000000 P 68e7a5c819f84c99a0f82b33e3f7eab0 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:43:28.387768 19679 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68e7a5c819f84c99a0f82b33e3f7eab0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 68e7a5c819f84c99a0f82b33e3f7eab0. Latest consensus state: current_term: 1 leader_uuid: "68e7a5c819f84c99a0f82b33e3f7eab0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68e7a5c819f84c99a0f82b33e3f7eab0" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 40499 } } }
I20260925 02:43:28.387786 19678 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68e7a5c819f84c99a0f82b33e3f7eab0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "68e7a5c819f84c99a0f82b33e3f7eab0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68e7a5c819f84c99a0f82b33e3f7eab0" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 40499 } } }
I20260925 02:43:28.387928 19678 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68e7a5c819f84c99a0f82b33e3f7eab0 [sys.catalog]: This master's current role is: LEADER
I20260925 02:43:28.387929 19679 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68e7a5c819f84c99a0f82b33e3f7eab0 [sys.catalog]: This master's current role is: LEADER
I20260925 02:43:28.388414 19686 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:43:28.391340 19686 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:43:28.397293 19686 catalog_manager.cc:1347] Generated new cluster ID: 70f1412c39d54cdc8dd6f0162edf2b87
I20260925 02:43:28.397362 19686 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 02:43:28.406569 19686 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 02:43:28.407584 19686 catalog_manager.cc:1504] Loading token signing keys...
I20260925 02:43:28.420660 19686 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 68e7a5c819f84c99a0f82b33e3f7eab0: Generated new TSK 0
I20260925 02:43:28.421551 19686 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.157.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.157.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.157.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.129.keytab.
Entry for principal kudu/127.13.157.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.129.keytab.
I20260925 02:43:28.457515 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790304178325258-13942-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790304178325258-13942-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790304178325258-13942-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790304178325258-13942-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.157.129:0
--local_ip_for_outbound_sockets=127.13.157.129
--webserver_interface=127.13.157.129
--webserver_port=0
--tserver_master_addrs=127.13.157.190:40499
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.129.keytab
--principal=kudu/127.13.157.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:42549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:43:28.567643 19698 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:43:28.567909 19698 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:43:28.567965 19698 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:43:28.571995 19698 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:43:28.572057 19698 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:43:28.572120 19698 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.157.129
I20260925 02:43:28.577406 19698 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:42549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790304178325258-13942-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790304178325258-13942-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.129.keytab
--principal=kudu/127.13.157.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.157.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790304178325258-13942-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.157.129
--webserver_port=0
--tserver_master_addrs=127.13.157.190:40499
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19698
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.157.129
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790304178325258-13942-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:43:28.578469 19698 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:43:28.579537 19698 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:43:28.589780 19704 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:28.589780 19706 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:28.589780 19703 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:43:28.590145 19698 server_base.cc:1034] running on GCE node
I20260925 02:43:28.590549 19698 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:43:28.591269 19698 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:43:28.592459 19698 hybrid_clock.cc:648] HybridClock initialized: now 1790304208592438 us; error 33 us; skew 500 ppm
Sep 25 02:43:28 dist-test-slave-kznw krb5kdc[19590](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304208, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:43:28.595304 19698 init.cc:377] Logged in from keytab as kudu/127.13.157.129@KRBTEST.COM (short username kudu)
I20260925 02:43:28.596211 19698 webserver.cc:460] Webserver started at http://127.13.157.129:45191/ using document root <none> and password file <none>
I20260925 02:43:28.596771 19698 fs_manager.cc:359] Metadata directory not provided
I20260925 02:43:28.596845 19698 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:43:28.597072 19698 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:43:28.599291 19698 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790304178325258-13942-0/minicluster-data/ts-0/data/instance:
uuid: "111200bd915649fdb6c7cdf5d9de23f7"
format_stamp: "Formatted at 2026-09-25 02:43:28 on dist-test-slave-kznw"
server_key: "0af03ab1cca61dfe98926df5b8e43891"
server_key_iv: "6c56320e76d4fec64469d1fe8af9e436"
server_key_version: "encryptionkey@0"
I20260925 02:43:28.599925 19698 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790304178325258-13942-0/minicluster-data/ts-0/wal/instance:
uuid: "111200bd915649fdb6c7cdf5d9de23f7"
format_stamp: "Formatted at 2026-09-25 02:43:28 on dist-test-slave-kznw"
server_key: "0af03ab1cca61dfe98926df5b8e43891"
server_key_iv: "6c56320e76d4fec64469d1fe8af9e436"
server_key_version: "encryptionkey@0"
I20260925 02:43:28.603880 19698 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260925 02:43:28.606514 19713 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:28.607755 19698 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260925 02:43:28.607859 19698 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790304178325258-13942-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790304178325258-13942-0/minicluster-data/ts-0/wal
uuid: "111200bd915649fdb6c7cdf5d9de23f7"
format_stamp: "Formatted at 2026-09-25 02:43:28 on dist-test-slave-kznw"
server_key: "0af03ab1cca61dfe98926df5b8e43891"
server_key_iv: "6c56320e76d4fec64469d1fe8af9e436"
server_key_version: "encryptionkey@0"
I20260925 02:43:28.607952 19698 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790304178325258-13942-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790304178325258-13942-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790304178325258-13942-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:43:28.628262 19698 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:43:28.631438 19698 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:43:28.631628 19698 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:43:28.632237 19698 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:43:28.633066 19698 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:43:28.633111 19698 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:28.633165 19698 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:43:28.633190 19698 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:28.641494 19698 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.129:34915
I20260925 02:43:28.641528 19829 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.129:34915 every 8 connection(s)
I20260925 02:43:28.642419 19698 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790304178325258-13942-0/minicluster-data/ts-0/data/info.pb
I20260925 02:43:28.643903 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 19698
I20260925 02:43:28.644054 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790304178325258-13942-0/minicluster-data/ts-0/wal/instance
I20260925 02:43:28.644425 13942 external_mini_cluster.cc:1428] Setting key 20da109be68c37d4b2b847df92ce12bb
Sep 25 02:43:28 dist-test-slave-kznw krb5kdc[19590](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304208, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.129@KRBTEST.COM for kudu/127.13.157.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.157.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.157.130@KRBTEST.COM" created.
I20260925 02:43:28.656636 19675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:28.644406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.129:38387 (local address 127.13.157.190:40499)
0925 02:43:28.644546 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:28.644551 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:28.646084 (+ 1533us) server_negotiation.cc:407] Connection header received
0925 02:43:28.647422 (+ 1338us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:28.647426 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:28.647478 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:28.647551 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:28.648695 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:28.649233 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:28.650377 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:28.650615 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:28.650703 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:28.653021 (+ 2318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:28.653025 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:28.653028 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:43:28.653065 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:28.654399 (+ 1334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:28.654925 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:28.654929 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:28.654930 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:28.655064 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:28.655411 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:28.655414 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:28.655416 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:28.655652 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:28.655750 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:28.656360 (+ 610us) server_negotiation.cc:299] Negotiation successful
0925 02:43:28.656484 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260925 02:43:28.657718 19832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:28.644948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:40499 (local address 127.13.157.129:38387)
0925 02:43:28.645919 (+ 971us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:28.645978 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:28.647221 (+ 1243us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:28.647727 (+ 506us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:28.647733 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:28.648074 (+ 341us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:28.648551 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:28.648565 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:28.649349 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:28.649353 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:28.650263 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:28.650273 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:28.650753 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:28.650757 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:28.650951 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:28.651490 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:28.651506 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:28.652867 (+ 1361us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:43:28.654527 (+ 1660us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:28.654533 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:28.654549 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:28.654804 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:28.655174 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:28.655177 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:28.655179 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:28.655321 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:28.655759 (+ 438us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:43:28.655766 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:43:28.656071 (+ 305us) client_negotiation.cc:769] Sending connection context
0925 02:43:28.656291 (+ 220us) client_negotiation.cc:240] Negotiation successful
0925 02:43:28.656673 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":738,"thread_start_us":142,"threads_started":1}
I20260925 02:43:28.658955 19830 heartbeater.cc:347] Connected to a master server at 127.13.157.190:40499
I20260925 02:43:28.659242 19830 heartbeater.cc:464] Registering TS with master...
I20260925 02:43:28.659731 19830 heartbeater.cc:511] Master 127.13.157.190:40499 requested a full tablet report, sending...
I20260925 02:43:28.661433 19635 ts_manager.cc:194] Registered new tserver with Master: 111200bd915649fdb6c7cdf5d9de23f7 (127.13.157.129:34915)
I20260925 02:43:28.663061 19635 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.157.129@KRBTEST.COM'} at 127.13.157.129:38387
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.157.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.130.keytab.
Entry for principal kudu/127.13.157.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.130.keytab.
I20260925 02:43:28.674043 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790304178325258-13942-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790304178325258-13942-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790304178325258-13942-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790304178325258-13942-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.157.130:0
--local_ip_for_outbound_sockets=127.13.157.130
--webserver_interface=127.13.157.130
--webserver_port=0
--tserver_master_addrs=127.13.157.190:40499
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.130.keytab
--principal=kudu/127.13.157.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:42549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:43:28.784870 19835 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:43:28.785157 19835 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:43:28.785202 19835 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:43:28.789376 19835 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:43:28.789438 19835 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:43:28.789495 19835 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.157.130
I20260925 02:43:28.794775 19835 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:42549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790304178325258-13942-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790304178325258-13942-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.130.keytab
--principal=kudu/127.13.157.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.157.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790304178325258-13942-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.157.130
--webserver_port=0
--tserver_master_addrs=127.13.157.190:40499
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19835
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.157.130
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790304178325258-13942-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:43:28.795819 19835 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:43:28.796787 19835 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:43:28.806499 19841 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:43:28.806481 19835 server_base.cc:1034] running on GCE node
W20260925 02:43:28.806466 19840 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:28.806489 19843 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:43:28.807114 19835 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:43:28.807763 19835 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:43:28.808923 19835 hybrid_clock.cc:648] HybridClock initialized: now 1790304208808905 us; error 32 us; skew 500 ppm
Sep 25 02:43:28 dist-test-slave-kznw krb5kdc[19590](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304208, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:43:28.811806 19835 init.cc:377] Logged in from keytab as kudu/127.13.157.130@KRBTEST.COM (short username kudu)
I20260925 02:43:28.812685 19835 webserver.cc:460] Webserver started at http://127.13.157.130:39923/ using document root <none> and password file <none>
I20260925 02:43:28.813212 19835 fs_manager.cc:359] Metadata directory not provided
I20260925 02:43:28.813279 19835 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:43:28.813472 19835 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:43:28.815604 19835 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790304178325258-13942-0/minicluster-data/ts-1/data/instance:
uuid: "2669781f7b6146509bc95316562441e0"
format_stamp: "Formatted at 2026-09-25 02:43:28 on dist-test-slave-kznw"
server_key: "b3be77ddf2bf91764b314e71f83e2170"
server_key_iv: "02d9db7849317e6f7b4181e1cab1cbe1"
server_key_version: "encryptionkey@0"
I20260925 02:43:28.816140 19835 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790304178325258-13942-0/minicluster-data/ts-1/wal/instance:
uuid: "2669781f7b6146509bc95316562441e0"
format_stamp: "Formatted at 2026-09-25 02:43:28 on dist-test-slave-kznw"
server_key: "b3be77ddf2bf91764b314e71f83e2170"
server_key_iv: "02d9db7849317e6f7b4181e1cab1cbe1"
server_key_version: "encryptionkey@0"
I20260925 02:43:28.819837 19835 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260925 02:43:28.822274 19850 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:28.824004 19835 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260925 02:43:28.824115 19835 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790304178325258-13942-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790304178325258-13942-0/minicluster-data/ts-1/wal
uuid: "2669781f7b6146509bc95316562441e0"
format_stamp: "Formatted at 2026-09-25 02:43:28 on dist-test-slave-kznw"
server_key: "b3be77ddf2bf91764b314e71f83e2170"
server_key_iv: "02d9db7849317e6f7b4181e1cab1cbe1"
server_key_version: "encryptionkey@0"
I20260925 02:43:28.824219 19835 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790304178325258-13942-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790304178325258-13942-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790304178325258-13942-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:43:28.846880 19835 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:43:28.850265 19835 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:43:28.850450 19835 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:43:28.851097 19835 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:43:28.851930 19835 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:43:28.851977 19835 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:28.852031 19835 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:43:28.852061 19835 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:28.860471 19835 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.130:37091
I20260925 02:43:28.860499 19967 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.130:37091 every 8 connection(s)
I20260925 02:43:28.861330 19835 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790304178325258-13942-0/minicluster-data/ts-1/data/info.pb
I20260925 02:43:28.870412 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 19835
I20260925 02:43:28.870548 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790304178325258-13942-0/minicluster-data/ts-1/wal/instance
I20260925 02:43:28.870865 13942 external_mini_cluster.cc:1428] Setting key 99945df7d895bb5c611b645bd2140b5a
Sep 25 02:43:28 dist-test-slave-kznw krb5kdc[19590](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304208, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.130@KRBTEST.COM for kudu/127.13.157.190@KRBTEST.COM
I20260925 02:43:28.875192 19675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:28.863153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.130:43893 (local address 127.13.157.190:40499)
0925 02:43:28.863290 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:28.863294 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:28.864123 (+ 829us) server_negotiation.cc:407] Connection header received
0925 02:43:28.865529 (+ 1406us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:28.865533 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:28.865599 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:28.865679 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:28.866969 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:28.867536 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:28.868728 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:28.868972 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:28.869069 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:28.871774 (+ 2705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:28.871777 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:28.871779 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:43:28.871805 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:28.873193 (+ 1388us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:28.873683 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:28.873686 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:28.873687 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:28.873768 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:28.874093 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:28.874097 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:28.874099 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:28.874281 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:28.874366 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:28.874873 (+ 507us) server_negotiation.cc:299] Negotiation successful
0925 02:43:28.875026 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260925 02:43:28.876027 19970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:28.863449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:40499 (local address 127.13.157.130:43893)
0925 02:43:28.863914 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:28.864007 (+ 93us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:28.865288 (+ 1281us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:28.865871 (+ 583us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:28.865878 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:28.866263 (+ 385us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:28.866767 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:28.866791 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:28.867673 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:28.867676 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:28.868593 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:28.868605 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:28.869136 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:28.869141 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:28.869303 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:28.869887 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:28.869904 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:28.871623 (+ 1719us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:43:28.873306 (+ 1683us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:28.873312 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:28.873329 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:28.873588 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:28.873849 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:28.873852 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:28.873854 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:28.874006 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:28.874378 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:43:28.874385 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:43:28.874640 (+ 255us) client_negotiation.cc:769] Sending connection context
0925 02:43:28.874813 (+ 173us) client_negotiation.cc:240] Negotiation successful
0925 02:43:28.875184 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":105,"threads_started":1}
I20260925 02:43:28.877250 19968 heartbeater.cc:347] Connected to a master server at 127.13.157.190:40499
I20260925 02:43:28.877537 19968 heartbeater.cc:464] Registering TS with master...
I20260925 02:43:28.878134 19968 heartbeater.cc:511] Master 127.13.157.190:40499 requested a full tablet report, sending...
I20260925 02:43:28.879429 19635 ts_manager.cc:194] Registered new tserver with Master: 2669781f7b6146509bc95316562441e0 (127.13.157.130:37091)
WARNING: no policy specified for kudu/127.13.157.131@KRBTEST.COM; defaulting to no policy
I20260925 02:43:28.880232 19635 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.157.130@KRBTEST.COM'} at 127.13.157.130:43893
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.157.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.157.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.131.keytab.
Entry for principal kudu/127.13.157.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.131.keytab.
I20260925 02:43:28.898742 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790304178325258-13942-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790304178325258-13942-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790304178325258-13942-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790304178325258-13942-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.157.131:0
--local_ip_for_outbound_sockets=127.13.157.131
--webserver_interface=127.13.157.131
--webserver_port=0
--tserver_master_addrs=127.13.157.190:40499
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.131.keytab
--principal=kudu/127.13.157.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:42549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:43:29.008584 19973 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:43:29.008831 19973 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:43:29.008874 19973 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:43:29.012907 19973 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:43:29.012956 19973 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:43:29.013015 19973 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.157.131
I20260925 02:43:29.018213 19973 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:42549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790304178325258-13942-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790304178325258-13942-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.131.keytab
--principal=kudu/127.13.157.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.157.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790304178325258-13942-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.157.131
--webserver_port=0
--tserver_master_addrs=127.13.157.190:40499
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19973
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.157.131
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790304178325258-13942-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:43:29.019258 19973 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:43:29.020260 19973 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:43:29.029596 19982 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:43:29.029637 19973 server_base.cc:1034] running on GCE node
W20260925 02:43:29.029603 19980 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:29.029668 19979 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:43:29.030246 19973 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:43:29.030812 19973 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:43:29.031965 19973 hybrid_clock.cc:648] HybridClock initialized: now 1790304209031947 us; error 31 us; skew 500 ppm
Sep 25 02:43:29 dist-test-slave-kznw krb5kdc[19590](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304209, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:43:29.034572 19973 init.cc:377] Logged in from keytab as kudu/127.13.157.131@KRBTEST.COM (short username kudu)
I20260925 02:43:29.035419 19973 webserver.cc:460] Webserver started at http://127.13.157.131:33327/ using document root <none> and password file <none>
I20260925 02:43:29.035928 19973 fs_manager.cc:359] Metadata directory not provided
I20260925 02:43:29.036008 19973 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:43:29.036252 19973 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:43:29.038204 19973 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790304178325258-13942-0/minicluster-data/ts-2/data/instance:
uuid: "f2ad35e3769d45bcb526157411bd2e71"
format_stamp: "Formatted at 2026-09-25 02:43:29 on dist-test-slave-kznw"
server_key: "9ea02da35a0bfdc16a0192e40695d8ae"
server_key_iv: "d9c32463caf3696b24c0fc8cc219e7cb"
server_key_version: "encryptionkey@0"
I20260925 02:43:29.038725 19973 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790304178325258-13942-0/minicluster-data/ts-2/wal/instance:
uuid: "f2ad35e3769d45bcb526157411bd2e71"
format_stamp: "Formatted at 2026-09-25 02:43:29 on dist-test-slave-kznw"
server_key: "9ea02da35a0bfdc16a0192e40695d8ae"
server_key_iv: "d9c32463caf3696b24c0fc8cc219e7cb"
server_key_version: "encryptionkey@0"
I20260925 02:43:29.042376 19973 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260925 02:43:29.044816 19990 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:29.046083 19973 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260925 02:43:29.046185 19973 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790304178325258-13942-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790304178325258-13942-0/minicluster-data/ts-2/wal
uuid: "f2ad35e3769d45bcb526157411bd2e71"
format_stamp: "Formatted at 2026-09-25 02:43:29 on dist-test-slave-kznw"
server_key: "9ea02da35a0bfdc16a0192e40695d8ae"
server_key_iv: "d9c32463caf3696b24c0fc8cc219e7cb"
server_key_version: "encryptionkey@0"
I20260925 02:43:29.046286 19973 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790304178325258-13942-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790304178325258-13942-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790304178325258-13942-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:43:29.063958 19973 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:43:29.067274 19973 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:43:29.067474 19973 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:43:29.068111 19973 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:43:29.068961 19973 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:43:29.069006 19973 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:29.069049 19973 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:43:29.069077 19973 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:29.077291 19973 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.131:44291
I20260925 02:43:29.077342 20103 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.131:44291 every 8 connection(s)
I20260925 02:43:29.078195 19973 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790304178325258-13942-0/minicluster-data/ts-2/data/info.pb
I20260925 02:43:29.085071 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 19973
I20260925 02:43:29.085199 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790304178325258-13942-0/minicluster-data/ts-2/wal/instance
I20260925 02:43:29.085481 13942 external_mini_cluster.cc:1428] Setting key b48a07897021d7eb402bb8ce2cbff284
Sep 25 02:43:29 dist-test-slave-kznw krb5kdc[19590](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304209, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.131@KRBTEST.COM for kudu/127.13.157.190@KRBTEST.COM
I20260925 02:43:29.092259 19675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:29.080044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.131:44395 (local address 127.13.157.190:40499)
0925 02:43:29.080245 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:29.080249 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:29.081345 (+ 1096us) server_negotiation.cc:407] Connection header received
0925 02:43:29.082746 (+ 1401us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:29.082749 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:29.082812 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:29.082884 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:29.084124 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:29.084661 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:29.085842 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:29.086115 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:29.086210 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:29.088997 (+ 2787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:29.089001 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:29.089003 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:43:29.089030 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:29.090302 (+ 1272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:29.090777 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:29.090780 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:29.090782 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:29.090860 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:29.091194 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:29.091197 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:29.091199 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:29.091383 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:29.091467 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:29.091966 (+ 499us) server_negotiation.cc:299] Negotiation successful
0925 02:43:29.092095 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113}
I20260925 02:43:29.093066 20106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:29.080435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:40499 (local address 127.13.157.131:44395)
0925 02:43:29.081118 (+ 683us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:29.081165 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:29.082523 (+ 1358us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:29.083117 (+ 594us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:29.083124 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:29.083483 (+ 359us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:29.083968 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:29.083982 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:29.084781 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:29.084785 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:29.085726 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:29.085736 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:29.086295 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:29.086300 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:29.086473 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:29.087063 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:29.087079 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:29.088835 (+ 1756us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:43:29.090409 (+ 1574us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:29.090415 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:29.090430 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:29.090684 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:29.090955 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:29.090958 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:29.090960 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:29.091107 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:29.091478 (+ 371us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:43:29.091486 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:43:29.091735 (+ 249us) client_negotiation.cc:769] Sending connection context
0925 02:43:29.091907 (+ 172us) client_negotiation.cc:240] Negotiation successful
0925 02:43:29.092251 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":136,"threads_started":1}
I20260925 02:43:29.094190 20104 heartbeater.cc:347] Connected to a master server at 127.13.157.190:40499
I20260925 02:43:29.094439 20104 heartbeater.cc:464] Registering TS with master...
I20260925 02:43:29.094938 20104 heartbeater.cc:511] Master 127.13.157.190:40499 requested a full tablet report, sending...
I20260925 02:43:29.096083 19635 ts_manager.cc:194] Registered new tserver with Master: f2ad35e3769d45bcb526157411bd2e71 (127.13.157.131:44291)
I20260925 02:43:29.096669 19635 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.157.131@KRBTEST.COM'} at 127.13.157.131:44395
I20260925 02:43:29.099978 13942 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790304178325258-13942-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790304178325258-13942-0/krb5kdc/test-admin.keytab.
Sep 25 02:43:29 dist-test-slave-kznw krb5kdc[19590](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304209, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:43:29.116459 13942 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260925 02:43:29.119444 20112 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:43:29.117939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40540 (local address 127.13.157.129:34915)
0925 02:43:29.118230 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:29.118245 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:29.118269 (+ 24us) server_negotiation.cc:407] Connection header received
0925 02:43:29.118429 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:29.118436 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:29.118639 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:29.118769 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:29.119263 (+ 494us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40540: BlockingRecv error: recv got EOF from 127.0.0.1:40540 (error 108)
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":101,"threads_started":1}
W20260925 02:43:29.119691 13942 init.cc:244] Error while opening credential cache '/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790304178325258-13942-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790304178325258-13942-0/krb5kdc/krb5cc)
Sep 25 02:43:29 dist-test-slave-kznw krb5kdc[19590](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304209, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:43:29.120656 13942 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 25 02:43:29 dist-test-slave-kznw krb5kdc[19590](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304209, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:43:29.121522 13942 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 25 02:43:29 dist-test-slave-kznw krb5kdc[19590](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304209, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.157.129@KRBTEST.COM
I20260925 02:43:29.131145 20112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:29.122383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40544 (local address 127.13.157.129:34915)
0925 02:43:29.122519 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:29.122522 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:29.122539 (+ 17us) server_negotiation.cc:407] Connection header received
0925 02:43:29.122651 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:29.122653 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:29.122692 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:29.122757 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:29.123354 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:29.124150 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:29.124991 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:29.125375 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:29.125464 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:29.127754 (+ 2290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:29.127763 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:29.127777 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:43:29.127802 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:29.129358 (+ 1556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:29.129850 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:29.129855 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:29.129857 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:29.129943 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:29.130207 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:29.130209 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:29.130210 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:29.130456 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:29.130617 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:29.130804 (+ 187us) server_negotiation.cc:299] Negotiation successful
0925 02:43:29.130984 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
W20260925 02:43:29.134982 20112 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:43:29.134028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40554 (local address 127.13.157.129:34915)
0925 02:43:29.134155 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:29.134159 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:29.134238 (+ 79us) server_negotiation.cc:407] Connection header received
0925 02:43:29.134327 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:29.134329 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:29.134364 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:29.134423 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:29.134855 (+ 432us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40554: BlockingRecv error: recv got EOF from 127.0.0.1:40554 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260925 02:43:29.135083 13942 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790304178325258-13942-0/krb5kdc/krb5cc' of type FILE
Sep 25 02:43:29 dist-test-slave-kznw krb5kdc[19590](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304209, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:43:29.136186 13942 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 25 02:43:29 dist-test-slave-kznw krb5kdc[19590](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304209, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:43:29.137056 13942 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 25 02:43:29 dist-test-slave-kznw krb5kdc[19590](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304209, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.157.129@KRBTEST.COM
I20260925 02:43:29.145002 20112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:29.137898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40568 (local address 127.13.157.129:34915)
0925 02:43:29.138062 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:29.138066 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:29.138078 (+ 12us) server_negotiation.cc:407] Connection header received
0925 02:43:29.138162 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:29.138164 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:29.138199 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:29.138260 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:29.138656 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:29.139245 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:29.139978 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:29.140237 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:29.140311 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:29.142353 (+ 2042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:29.142356 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:29.142358 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:43:29.142380 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:29.143673 (+ 1293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:29.144044 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:29.144047 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:29.144048 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:29.144132 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:29.144363 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:29.144366 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:29.144367 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:29.144526 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:29.144623 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:29.144768 (+ 145us) server_negotiation.cc:299] Negotiation successful
0925 02:43:29.144852 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
W20260925 02:43:29.147459 20112 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:43:29.146629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40570 (local address 127.13.157.129:34915)
0925 02:43:29.146748 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:29.146752 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:29.146819 (+ 67us) server_negotiation.cc:407] Connection header received
0925 02:43:29.146895 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:29.146897 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:29.146964 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:29.147086 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:29.147368 (+ 282us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40570: BlockingRecv error: recv got EOF from 127.0.0.1:40570 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260925 02:43:29.147708 13942 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790304178325258-13942-0/krb5kdc/krb5cc' of type FILE
Sep 25 02:43:29 dist-test-slave-kznw krb5kdc[19590](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304209, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:43:29.148595 13942 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 25 02:43:29 dist-test-slave-kznw krb5kdc[19590](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304209, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:43:29.149367 13942 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 25 02:43:29 dist-test-slave-kznw krb5kdc[19590](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304209, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.157.129@KRBTEST.COM
I20260925 02:43:29.157166 20112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:29.150194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40584 (local address 127.13.157.129:34915)
0925 02:43:29.150321 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:29.150325 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:29.150393 (+ 68us) server_negotiation.cc:407] Connection header received
0925 02:43:29.150493 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:29.150496 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:29.150533 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:29.150600 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:29.151076 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:29.151581 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:29.152308 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:29.152512 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:29.152605 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:29.154533 (+ 1928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:29.154537 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:29.154538 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:43:29.154575 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:29.155859 (+ 1284us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:29.156258 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:29.156261 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:29.156262 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:29.156326 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:29.156544 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:29.156556 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:29.156557 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:29.156695 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:29.156782 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:29.156974 (+ 192us) server_negotiation.cc:299] Negotiation successful
0925 02:43:29.157058 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260925 02:43:29.158037 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 19698
I20260925 02:43:29.163190 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 19835
I20260925 02:43:29.167892 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 19973
I20260925 02:43:29.172714 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 19603
2026-09-25T02:43:29Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2077 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 02:43:29 dist-test-slave-kznw krb5kdc[20127](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 02:43:29 dist-test-slave-kznw krb5kdc[20127](info): set up 2 sockets
Sep 25 02:43:29 dist-test-slave-kznw krb5kdc[20127](info): commencing operation
krb5kdc: starting...
W20260925 02:43:30.142542 13942 mini_kdc.cc:121] Time spent starting KDC: real 0.949s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/krb5kdc/test-user.keytab.
Sep 25 02:43:30 dist-test-slave-kznw krb5kdc[20127](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304210, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T02:43:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T02:43:30Z Disabled control of system clock
WARNING: no policy specified for oryx/127.13.157.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.157.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.157.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/krb5kdc/oryx_127.13.157.190.keytab.
Entry for principal oryx/127.13.157.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/krb5kdc/oryx_127.13.157.190.keytab.
I20260925 02:43:30.265302 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.157.190:34721
--webserver_interface=127.13.157.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/krb5kdc/oryx_127.13.157.190.keytab
--principal=oryx/127.13.157.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:46123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.157.190:34721
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:43:30.374341 20140 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:43:30.374609 20140 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:43:30.374675 20140 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:43:30.378675 20140 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:43:30.378738 20140 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:43:30.378764 20140 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:43:30.378785 20140 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:43:30.378805 20140 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:43:30.384245 20140 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:46123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.157.190:34721
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/krb5kdc/oryx_127.13.157.190.keytab
--principal=oryx/127.13.157.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.157.190:34721
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.157.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20140
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:43:30.385385 20140 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:43:30.386413 20140 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:43:30.394759 20148 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:30.394944 20145 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:43:30.394793 20140 server_base.cc:1034] running on GCE node
W20260925 02:43:30.395000 20146 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:43:30.395663 20140 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:43:30.396800 20140 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:43:30.397959 20140 hybrid_clock.cc:648] HybridClock initialized: now 1790304210397942 us; error 29 us; skew 500 ppm
Sep 25 02:43:30 dist-test-slave-kznw krb5kdc[20127](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304210, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.157.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:43:30.406610 20140 init.cc:377] Logged in from keytab as oryx/127.13.157.190@KRBTEST.COM (short username oryx)
I20260925 02:43:30.407603 20140 webserver.cc:460] Webserver started at http://127.13.157.190:43365/ using document root <none> and password file <none>
I20260925 02:43:30.408143 20140 fs_manager.cc:359] Metadata directory not provided
I20260925 02:43:30.408205 20140 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:43:30.408370 20140 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:43:30.410561 20140 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/master-0/data/instance:
uuid: "2b4937775ce34bed9ca35afdca0243b0"
format_stamp: "Formatted at 2026-09-25 02:43:30 on dist-test-slave-kznw"
server_key: "8e9b4c75ff452b6d9cbe1627070a404a"
server_key_iv: "ee7c367e17f6e7e60025f2f0d0004daf"
server_key_version: "encryptionkey@0"
I20260925 02:43:30.411237 20140 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/master-0/wal/instance:
uuid: "2b4937775ce34bed9ca35afdca0243b0"
format_stamp: "Formatted at 2026-09-25 02:43:30 on dist-test-slave-kznw"
server_key: "8e9b4c75ff452b6d9cbe1627070a404a"
server_key_iv: "ee7c367e17f6e7e60025f2f0d0004daf"
server_key_version: "encryptionkey@0"
I20260925 02:43:30.415068 20140 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260925 02:43:30.417522 20155 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:30.418885 20140 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260925 02:43:30.419040 20140 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/master-0/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/master-0/wal
uuid: "2b4937775ce34bed9ca35afdca0243b0"
format_stamp: "Formatted at 2026-09-25 02:43:30 on dist-test-slave-kznw"
server_key: "8e9b4c75ff452b6d9cbe1627070a404a"
server_key_iv: "ee7c367e17f6e7e60025f2f0d0004daf"
server_key_version: "encryptionkey@0"
I20260925 02:43:30.419147 20140 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:43:30.459530 20140 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:43:30.462683 20140 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:43:30.462860 20140 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:43:30.469786 20140 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.190:34721
I20260925 02:43:30.470052 20207 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.190:34721 every 8 connection(s)
I20260925 02:43:30.470798 20140 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
I20260925 02:43:30.471773 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 20140
I20260925 02:43:30.471906 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/master-0/wal/instance
I20260925 02:43:30.472240 13942 external_mini_cluster.cc:1428] Setting key a4b1665fd56f0147b6943c0d2d206a60
I20260925 02:43:30.474169 20208 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:43:30.480540 20208 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b4937775ce34bed9ca35afdca0243b0: Bootstrap starting.
Sep 25 02:43:30 dist-test-slave-kznw krb5kdc[20127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304210, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM
I20260925 02:43:30.482937 20208 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2b4937775ce34bed9ca35afdca0243b0: Neither blocks nor log segments found. Creating new log.
I20260925 02:43:30.483630 20208 log.cc:824] T 00000000000000000000000000000000 P 2b4937775ce34bed9ca35afdca0243b0: Log is configured to *not* fsync() on all Append() calls
I20260925 02:43:30.485538 20208 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b4937775ce34bed9ca35afdca0243b0: No bootstrap required, opened a new log
I20260925 02:43:30.486248 20211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:30.473696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42392 (local address 127.13.157.190:34721)
0925 02:43:30.474100 (+ 404us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:30.474116 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:30.474153 (+ 37us) server_negotiation.cc:407] Connection header received
0925 02:43:30.474818 (+ 665us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:30.474835 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:30.475224 (+ 389us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:30.475515 (+ 291us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:30.476351 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:30.477352 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:30.478464 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:30.479011 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:30.479131 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:30.481807 (+ 2676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:30.481812 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:30.481824 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:43:30.481852 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:30.483551 (+ 1699us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:30.483991 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:30.483997 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:30.484003 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:30.484130 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:30.484394 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:30.484397 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:30.484398 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:30.484774 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:30.484949 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:30.485235 (+ 286us) server_negotiation.cc:299] Negotiation successful
0925 02:43:30.485505 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":257,"thread_start_us":109,"threads_started":1}
I20260925 02:43:30.489023 20208 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2b4937775ce34bed9ca35afdca0243b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b4937775ce34bed9ca35afdca0243b0" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 34721 } }
I20260925 02:43:30.489233 20208 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2b4937775ce34bed9ca35afdca0243b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:43:30.489288 20208 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2b4937775ce34bed9ca35afdca0243b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b4937775ce34bed9ca35afdca0243b0, State: Initialized, Role: FOLLOWER
I20260925 02:43:30.489678 20208 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2b4937775ce34bed9ca35afdca0243b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b4937775ce34bed9ca35afdca0243b0" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 34721 } }
I20260925 02:43:30.489804 20208 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2b4937775ce34bed9ca35afdca0243b0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 02:43:30.489877 20208 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2b4937775ce34bed9ca35afdca0243b0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 02:43:30.489985 20208 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2b4937775ce34bed9ca35afdca0243b0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:43:30.490918 20208 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2b4937775ce34bed9ca35afdca0243b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b4937775ce34bed9ca35afdca0243b0" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 34721 } }
I20260925 02:43:30.491273 20208 leader_election.cc:302] T 00000000000000000000000000000000 P 2b4937775ce34bed9ca35afdca0243b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2b4937775ce34bed9ca35afdca0243b0; no voters:
I20260925 02:43:30.491585 20208 leader_election.cc:288] T 00000000000000000000000000000000 P 2b4937775ce34bed9ca35afdca0243b0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 02:43:30.491662 20213 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2b4937775ce34bed9ca35afdca0243b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:43:30.491881 20213 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2b4937775ce34bed9ca35afdca0243b0 [term 1 LEADER]: Becoming Leader. State: Replica: 2b4937775ce34bed9ca35afdca0243b0, State: Running, Role: LEADER
I20260925 02:43:30.492187 20213 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2b4937775ce34bed9ca35afdca0243b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b4937775ce34bed9ca35afdca0243b0" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 34721 } }
I20260925 02:43:30.492692 20208 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2b4937775ce34bed9ca35afdca0243b0 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:43:30.493700 20215 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b4937775ce34bed9ca35afdca0243b0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2b4937775ce34bed9ca35afdca0243b0. Latest consensus state: current_term: 1 leader_uuid: "2b4937775ce34bed9ca35afdca0243b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b4937775ce34bed9ca35afdca0243b0" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 34721 } } }
I20260925 02:43:30.493831 20215 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b4937775ce34bed9ca35afdca0243b0 [sys.catalog]: This master's current role is: LEADER
I20260925 02:43:30.494091 20214 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b4937775ce34bed9ca35afdca0243b0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2b4937775ce34bed9ca35afdca0243b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b4937775ce34bed9ca35afdca0243b0" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 34721 } } }
I20260925 02:43:30.494169 20214 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b4937775ce34bed9ca35afdca0243b0 [sys.catalog]: This master's current role is: LEADER
I20260925 02:43:30.494215 20222 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:43:30.497226 20222 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:43:30.502873 20222 catalog_manager.cc:1347] Generated new cluster ID: 19dd28aa2b484a96bcdd464cb2d07319
I20260925 02:43:30.502972 20222 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 02:43:30.514046 20222 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 02:43:30.515131 20222 catalog_manager.cc:1504] Loading token signing keys...
I20260925 02:43:30.521678 20222 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2b4937775ce34bed9ca35afdca0243b0: Generated new TSK 0
I20260925 02:43:30.522326 20222 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.13.157.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.157.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.157.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/krb5kdc/oryx_127.13.157.129.keytab.
Entry for principal oryx/127.13.157.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/krb5kdc/oryx_127.13.157.129.keytab.
I20260925 02:43:30.558993 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.157.129:0
--local_ip_for_outbound_sockets=127.13.157.129
--webserver_interface=127.13.157.129
--webserver_port=0
--tserver_master_addrs=127.13.157.190:34721
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/krb5kdc/oryx_127.13.157.129.keytab
--principal=oryx/127.13.157.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:46123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:43:30.677886 20234 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:43:30.678133 20234 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:43:30.678174 20234 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:43:30.682102 20234 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:43:30.682156 20234 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:43:30.682211 20234 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.157.129
I20260925 02:43:30.687819 20234 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:46123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/krb5kdc/oryx_127.13.157.129.keytab
--principal=oryx/127.13.157.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.157.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.157.129
--webserver_port=0
--tserver_master_addrs=127.13.157.190:34721
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20234
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.157.129
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:43:30.688791 20234 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:43:30.689755 20234 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:43:30.699347 20239 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:43:30.699437 20234 server_base.cc:1034] running on GCE node
W20260925 02:43:30.699385 20243 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:30.699360 20240 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:43:30.699961 20234 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:43:30.700609 20234 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:43:30.701778 20234 hybrid_clock.cc:648] HybridClock initialized: now 1790304210701761 us; error 29 us; skew 500 ppm
Sep 25 02:43:30 dist-test-slave-kznw krb5kdc[20127](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304210, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.157.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:43:30.704546 20234 init.cc:377] Logged in from keytab as oryx/127.13.157.129@KRBTEST.COM (short username oryx)
I20260925 02:43:30.705433 20234 webserver.cc:460] Webserver started at http://127.13.157.129:39687/ using document root <none> and password file <none>
I20260925 02:43:30.705978 20234 fs_manager.cc:359] Metadata directory not provided
I20260925 02:43:30.706060 20234 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:43:30.706257 20234 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:43:30.708256 20234 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/ts-0/data/instance:
uuid: "15b827f1b5534fafa26e3142b9e9d192"
format_stamp: "Formatted at 2026-09-25 02:43:30 on dist-test-slave-kznw"
server_key: "54e5127bd46f635a98e0f1a79eb3340f"
server_key_iv: "19ff5e85c1d80525f659aa5ec72632b8"
server_key_version: "encryptionkey@0"
I20260925 02:43:30.708791 20234 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/ts-0/wal/instance:
uuid: "15b827f1b5534fafa26e3142b9e9d192"
format_stamp: "Formatted at 2026-09-25 02:43:30 on dist-test-slave-kznw"
server_key: "54e5127bd46f635a98e0f1a79eb3340f"
server_key_iv: "19ff5e85c1d80525f659aa5ec72632b8"
server_key_version: "encryptionkey@0"
I20260925 02:43:30.712303 20234 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260925 02:43:30.714685 20252 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:30.715934 20234 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260925 02:43:30.716039 20234 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/ts-0/wal
uuid: "15b827f1b5534fafa26e3142b9e9d192"
format_stamp: "Formatted at 2026-09-25 02:43:30 on dist-test-slave-kznw"
server_key: "54e5127bd46f635a98e0f1a79eb3340f"
server_key_iv: "19ff5e85c1d80525f659aa5ec72632b8"
server_key_version: "encryptionkey@0"
I20260925 02:43:30.716132 20234 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:43:30.731781 20234 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:43:30.734586 20234 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:43:30.734740 20234 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:43:30.735363 20234 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:43:30.736157 20234 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:43:30.736200 20234 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:30.736238 20234 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:43:30.736258 20234 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:30.744400 20234 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.129:46073
I20260925 02:43:30.744460 20365 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.129:46073 every 8 connection(s)
I20260925 02:43:30.745275 20234 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/ts-0/data/info.pb
I20260925 02:43:30.745602 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 20234
I20260925 02:43:30.745738 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/ts-0/wal/instance
I20260925 02:43:30.746202 13942 external_mini_cluster.cc:1428] Setting key 7ecf3851fe454970b2cadb8db4991e25
Sep 25 02:43:30 dist-test-slave-kznw krb5kdc[20127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304210, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.157.129@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM
WARNING: no policy specified for oryx/127.13.157.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.157.130@KRBTEST.COM" created.
I20260925 02:43:30.759816 20211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:30.747202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.129:51177 (local address 127.13.157.190:34721)
0925 02:43:30.747332 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:30.747337 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:30.748395 (+ 1058us) server_negotiation.cc:407] Connection header received
0925 02:43:30.750125 (+ 1730us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:30.750128 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:30.750178 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:30.750253 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:30.751526 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:30.752073 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:30.753300 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:30.753540 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:30.753631 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:30.756148 (+ 2517us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:30.756152 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:30.756155 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:43:30.756195 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:30.757513 (+ 1318us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:30.758037 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:30.758040 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:30.758042 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:30.758149 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:30.758555 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:30.758559 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:30.758560 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:30.758786 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:30.758880 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:30.759478 (+ 598us) server_negotiation.cc:299] Negotiation successful
0925 02:43:30.759626 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260925 02:43:30.760670 20368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:30.747631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:34721 (local address 127.13.157.129:51177)
0925 02:43:30.748246 (+ 615us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:30.748303 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:30.749873 (+ 1570us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:30.750440 (+ 567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:30.750450 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:30.750846 (+ 396us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:30.751389 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:30.751404 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:30.752209 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:30.752213 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:30.753182 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:30.753193 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:30.753691 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:30.753695 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:30.753853 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:30.754386 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:30.754402 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:30.755924 (+ 1522us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:43:30.757622 (+ 1698us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:30.757627 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:30.757639 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:30.757892 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:30.758249 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:30.758251 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:30.758253 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:30.758420 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:30.758891 (+ 471us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:43:30.758898 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:43:30.759236 (+ 338us) client_negotiation.cc:769] Sending connection context
0925 02:43:30.759416 (+ 180us) client_negotiation.cc:240] Negotiation successful
0925 02:43:30.759781 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":152,"threads_started":1}
I20260925 02:43:30.761893 20366 heartbeater.cc:347] Connected to a master server at 127.13.157.190:34721
I20260925 02:43:30.762144 20366 heartbeater.cc:464] Registering TS with master...
I20260925 02:43:30.762621 20366 heartbeater.cc:511] Master 127.13.157.190:34721 requested a full tablet report, sending...
I20260925 02:43:30.764428 20171 ts_manager.cc:194] Registered new tserver with Master: 15b827f1b5534fafa26e3142b9e9d192 (127.13.157.129:46073)
I20260925 02:43:30.766072 20171 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.157.129@KRBTEST.COM'} at 127.13.157.129:51177
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.157.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/krb5kdc/oryx_127.13.157.130.keytab.
Entry for principal oryx/127.13.157.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/krb5kdc/oryx_127.13.157.130.keytab.
I20260925 02:43:30.775874 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.157.130:0
--local_ip_for_outbound_sockets=127.13.157.130
--webserver_interface=127.13.157.130
--webserver_port=0
--tserver_master_addrs=127.13.157.190:34721
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/krb5kdc/oryx_127.13.157.130.keytab
--principal=oryx/127.13.157.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:46123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:43:30.885504 20371 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:43:30.885774 20371 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:43:30.885833 20371 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:43:30.889859 20371 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:43:30.889930 20371 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:43:30.890012 20371 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.157.130
I20260925 02:43:30.895430 20371 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:46123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/krb5kdc/oryx_127.13.157.130.keytab
--principal=oryx/127.13.157.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.157.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.157.130
--webserver_port=0
--tserver_master_addrs=127.13.157.190:34721
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20371
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.157.130
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:43:30.896476 20371 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:43:30.897495 20371 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:43:30.906973 20376 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:30.906987 20377 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:43:30.907258 20371 server_base.cc:1034] running on GCE node
W20260925 02:43:30.906987 20379 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:43:30.907758 20371 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:43:30.908406 20371 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:43:30.909564 20371 hybrid_clock.cc:648] HybridClock initialized: now 1790304210909545 us; error 29 us; skew 500 ppm
Sep 25 02:43:30 dist-test-slave-kznw krb5kdc[20127](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304210, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.157.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:43:30.912220 20371 init.cc:377] Logged in from keytab as oryx/127.13.157.130@KRBTEST.COM (short username oryx)
I20260925 02:43:30.913100 20371 webserver.cc:460] Webserver started at http://127.13.157.130:37403/ using document root <none> and password file <none>
I20260925 02:43:30.913619 20371 fs_manager.cc:359] Metadata directory not provided
I20260925 02:43:30.913684 20371 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:43:30.913888 20371 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:43:30.916002 20371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/ts-1/data/instance:
uuid: "03b8c1c27f5b471498f53d8d326b61db"
format_stamp: "Formatted at 2026-09-25 02:43:30 on dist-test-slave-kznw"
server_key: "1ae251b5050bb94b14cce9f651bc000a"
server_key_iv: "d3b3727d1341d3b77446586d4e259bbd"
server_key_version: "encryptionkey@0"
I20260925 02:43:30.916544 20371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/ts-1/wal/instance:
uuid: "03b8c1c27f5b471498f53d8d326b61db"
format_stamp: "Formatted at 2026-09-25 02:43:30 on dist-test-slave-kznw"
server_key: "1ae251b5050bb94b14cce9f651bc000a"
server_key_iv: "d3b3727d1341d3b77446586d4e259bbd"
server_key_version: "encryptionkey@0"
I20260925 02:43:30.920256 20371 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260925 02:43:30.922641 20386 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:30.923955 20371 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260925 02:43:30.924055 20371 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/ts-1/wal
uuid: "03b8c1c27f5b471498f53d8d326b61db"
format_stamp: "Formatted at 2026-09-25 02:43:30 on dist-test-slave-kznw"
server_key: "1ae251b5050bb94b14cce9f651bc000a"
server_key_iv: "d3b3727d1341d3b77446586d4e259bbd"
server_key_version: "encryptionkey@0"
I20260925 02:43:30.924142 20371 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:43:30.944737 20371 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:43:30.947674 20371 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:43:30.947824 20371 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:43:30.948369 20371 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:43:30.949146 20371 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:43:30.949189 20371 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:30.949229 20371 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:43:30.949249 20371 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:30.957696 20371 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.130:36015
I20260925 02:43:30.957726 20501 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.130:36015 every 8 connection(s)
I20260925 02:43:30.958634 20371 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/ts-1/data/info.pb
I20260925 02:43:30.962298 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 20371
I20260925 02:43:30.962462 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/ts-1/wal/instance
I20260925 02:43:30.962823 13942 external_mini_cluster.cc:1428] Setting key 30c87b9f2f2193613ee6c3dc7b962a20
Sep 25 02:43:30 dist-test-slave-kznw krb5kdc[20127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304210, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.157.130@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM
WARNING: no policy specified for oryx/127.13.157.131@KRBTEST.COM; defaulting to no policy
I20260925 02:43:30.973461 20211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:30.960469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.130:36889 (local address 127.13.157.190:34721)
0925 02:43:30.960611 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:30.960615 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:30.961503 (+ 888us) server_negotiation.cc:407] Connection header received
0925 02:43:30.963203 (+ 1700us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:30.963205 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:30.963255 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:30.963347 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:30.964796 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:30.965316 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:30.966736 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:30.967088 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:30.967193 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:30.970000 (+ 2807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:30.970004 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:30.970005 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:43:30.970030 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:30.971337 (+ 1307us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:30.971922 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:30.971924 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:30.971925 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:30.971991 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:30.972362 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:30.972364 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:30.972365 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:30.972524 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:30.972612 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:30.973242 (+ 630us) server_negotiation.cc:299] Negotiation successful
0925 02:43:30.973339 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260925 02:43:30.974421 20504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:30.960835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:34721 (local address 127.13.157.130:36889)
0925 02:43:30.961328 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:30.961380 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:30.962972 (+ 1592us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:30.963520 (+ 548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:30.963529 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:30.963937 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:30.964637 (+ 700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:30.964659 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:30.965438 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:30.965442 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:30.966607 (+ 1165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:30.966619 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:30.967239 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:30.967244 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:30.967443 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:30.968090 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:30.968113 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:30.969818 (+ 1705us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:43:30.971457 (+ 1639us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:30.971463 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:30.971479 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:30.971816 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:30.972070 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:30.972074 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:30.972076 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:30.972268 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:30.972640 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:43:30.972647 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:43:30.972979 (+ 332us) client_negotiation.cc:769] Sending connection context
0925 02:43:30.973198 (+ 219us) client_negotiation.cc:240] Negotiation successful
0925 02:43:30.973458 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":123,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.157.131@KRBTEST.COM" created.
I20260925 02:43:30.975660 20502 heartbeater.cc:347] Connected to a master server at 127.13.157.190:34721
I20260925 02:43:30.975891 20502 heartbeater.cc:464] Registering TS with master...
I20260925 02:43:30.976351 20502 heartbeater.cc:511] Master 127.13.157.190:34721 requested a full tablet report, sending...
I20260925 02:43:30.977483 20171 ts_manager.cc:194] Registered new tserver with Master: 03b8c1c27f5b471498f53d8d326b61db (127.13.157.130:36015)
I20260925 02:43:30.978056 20171 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.157.130@KRBTEST.COM'} at 127.13.157.130:36889
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.157.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/krb5kdc/oryx_127.13.157.131.keytab.
Entry for principal oryx/127.13.157.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/krb5kdc/oryx_127.13.157.131.keytab.
I20260925 02:43:30.990018 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.157.131:0
--local_ip_for_outbound_sockets=127.13.157.131
--webserver_interface=127.13.157.131
--webserver_port=0
--tserver_master_addrs=127.13.157.190:34721
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/krb5kdc/oryx_127.13.157.131.keytab
--principal=oryx/127.13.157.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:46123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:43:31.099018 20507 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:43:31.099284 20507 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:43:31.099327 20507 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:43:31.103410 20507 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:43:31.103474 20507 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:43:31.103541 20507 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.157.131
I20260925 02:43:31.109018 20507 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:46123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/krb5kdc/oryx_127.13.157.131.keytab
--principal=oryx/127.13.157.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.157.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.157.131
--webserver_port=0
--tserver_master_addrs=127.13.157.190:34721
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20507
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.157.131
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:43:31.110100 20507 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:43:31.111181 20507 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:43:31.120914 20512 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:31.120923 20515 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:43:31.121039 20507 server_base.cc:1034] running on GCE node
W20260925 02:43:31.121300 20513 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:43:31.121690 20507 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:43:31.122352 20507 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:43:31.123528 20507 hybrid_clock.cc:648] HybridClock initialized: now 1790304211123485 us; error 56 us; skew 500 ppm
Sep 25 02:43:31 dist-test-slave-kznw krb5kdc[20127](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304211, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.157.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:43:31.126288 20507 init.cc:377] Logged in from keytab as oryx/127.13.157.131@KRBTEST.COM (short username oryx)
I20260925 02:43:31.127236 20507 webserver.cc:460] Webserver started at http://127.13.157.131:37925/ using document root <none> and password file <none>
I20260925 02:43:31.127812 20507 fs_manager.cc:359] Metadata directory not provided
I20260925 02:43:31.127883 20507 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:43:31.128113 20507 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:43:31.130136 20507 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/ts-2/data/instance:
uuid: "323109284d62402a989a4cd4f8ec3c54"
format_stamp: "Formatted at 2026-09-25 02:43:31 on dist-test-slave-kznw"
server_key: "c5d3d6a1ceb9ec3c736634081de6ef39"
server_key_iv: "83ee8d4d33949feb1c59c6024e4fe419"
server_key_version: "encryptionkey@0"
I20260925 02:43:31.130698 20507 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/ts-2/wal/instance:
uuid: "323109284d62402a989a4cd4f8ec3c54"
format_stamp: "Formatted at 2026-09-25 02:43:31 on dist-test-slave-kznw"
server_key: "c5d3d6a1ceb9ec3c736634081de6ef39"
server_key_iv: "83ee8d4d33949feb1c59c6024e4fe419"
server_key_version: "encryptionkey@0"
I20260925 02:43:31.134467 20507 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260925 02:43:31.136941 20522 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:31.138242 20507 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260925 02:43:31.138358 20507 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/ts-2/wal
uuid: "323109284d62402a989a4cd4f8ec3c54"
format_stamp: "Formatted at 2026-09-25 02:43:31 on dist-test-slave-kznw"
server_key: "c5d3d6a1ceb9ec3c736634081de6ef39"
server_key_iv: "83ee8d4d33949feb1c59c6024e4fe419"
server_key_version: "encryptionkey@0"
I20260925 02:43:31.138456 20507 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:43:31.155028 20507 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:43:31.158010 20507 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:43:31.158208 20507 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:43:31.158835 20507 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:43:31.159732 20507 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:43:31.159782 20507 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:31.159835 20507 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:43:31.159864 20507 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:31.168423 20507 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.131:33075
I20260925 02:43:31.168478 20639 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.131:33075 every 8 connection(s)
I20260925 02:43:31.169327 20507 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/ts-2/data/info.pb
I20260925 02:43:31.175935 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 20507
I20260925 02:43:31.176129 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/ts-2/wal/instance
I20260925 02:43:31.176450 13942 external_mini_cluster.cc:1428] Setting key eff9fc8be493c616594c1e2237ccc513
Sep 25 02:43:31 dist-test-slave-kznw krb5kdc[20127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304211, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.157.131@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM
I20260925 02:43:31.183132 20211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:31.171068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.131:38279 (local address 127.13.157.190:34721)
0925 02:43:31.171222 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:31.171227 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:31.172051 (+ 824us) server_negotiation.cc:407] Connection header received
0925 02:43:31.173368 (+ 1317us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:31.173371 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:31.173422 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:31.173499 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:31.174764 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:31.175343 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:31.176848 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:31.177158 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:31.177267 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:31.179950 (+ 2683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:31.179953 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:31.179955 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:43:31.179980 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:31.181338 (+ 1358us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:31.181777 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:31.181779 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:31.181780 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:31.181848 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:31.182098 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:31.182101 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:31.182102 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:31.182262 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:31.182335 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:31.182878 (+ 543us) server_negotiation.cc:299] Negotiation successful
0925 02:43:31.183005 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260925 02:43:31.183983 20642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:31.171448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:34721 (local address 127.13.157.131:38279)
0925 02:43:31.171892 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:31.171937 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:31.173153 (+ 1216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:31.173660 (+ 507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:31.173667 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:31.174099 (+ 432us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:31.174619 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:31.174635 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:31.175457 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:31.175460 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:31.176721 (+ 1261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:31.176732 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:31.177344 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:31.177349 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:31.177561 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:31.178216 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:31.178232 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:31.179799 (+ 1567us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:43:31.181443 (+ 1644us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:31.181448 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:31.181459 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:31.181705 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:31.181908 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:31.181910 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:31.181911 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:31.182031 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:31.182383 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:43:31.182389 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:43:31.182646 (+ 257us) client_negotiation.cc:769] Sending connection context
0925 02:43:31.182830 (+ 184us) client_negotiation.cc:240] Negotiation successful
0925 02:43:31.183139 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":120,"threads_started":1}
I20260925 02:43:31.185098 20640 heartbeater.cc:347] Connected to a master server at 127.13.157.190:34721
I20260925 02:43:31.185315 20640 heartbeater.cc:464] Registering TS with master...
I20260925 02:43:31.185792 20640 heartbeater.cc:511] Master 127.13.157.190:34721 requested a full tablet report, sending...
I20260925 02:43:31.186817 20171 ts_manager.cc:194] Registered new tserver with Master: 323109284d62402a989a4cd4f8ec3c54 (127.13.157.131:33075)
I20260925 02:43:31.187394 20171 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.157.131@KRBTEST.COM'} at 127.13.157.131:38279
I20260925 02:43:31.190419 13942 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.13.157.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.157.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.157.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/krb5kdc/oryx_127.13.157.189.keytab.
Entry for principal oryx/127.13.157.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/krb5kdc/oryx_127.13.157.189.keytab.
I20260925 02:43:31.217484 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.157.189:33117
--webserver_interface=127.13.157.189
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/krb5kdc/oryx_127.13.157.189.keytab
--principal=oryx/127.13.157.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:46123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.157.190:34721,127.13.157.189:33117
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:43:31.327332 20645 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:43:31.327602 20645 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:43:31.327659 20645 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:43:31.331656 20645 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:43:31.331715 20645 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:43:31.331743 20645 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:43:31.331763 20645 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:43:31.331784 20645 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:43:31.336951 20645 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:46123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.13.157.190:34721,127.13.157.189:33117
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/krb5kdc/oryx_127.13.157.189.keytab
--principal=oryx/127.13.157.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.157.189:33117
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.13.157.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20645
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
W20260925 02:43:31.337195 20645 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.13.157.190:34721,127.13.157.189:33117'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260925 02:43:31.338078 20645 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:43:31.339178 20645 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:43:31.347976 20653 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:31.347986 20650 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:31.347986 20651 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:43:31.348380 20645 server_base.cc:1034] running on GCE node
I20260925 02:43:31.348903 20645 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:43:31.350018 20645 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:43:31.351176 20645 hybrid_clock.cc:648] HybridClock initialized: now 1790304211351149 us; error 40 us; skew 500 ppm
Sep 25 02:43:31 dist-test-slave-kznw krb5kdc[20127](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304211, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.157.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:43:31.353878 20645 init.cc:377] Logged in from keytab as oryx/127.13.157.189@KRBTEST.COM (short username oryx)
I20260925 02:43:31.354758 20645 webserver.cc:460] Webserver started at http://127.13.157.189:34963/ using document root <none> and password file <none>
I20260925 02:43:31.355326 20645 fs_manager.cc:359] Metadata directory not provided
I20260925 02:43:31.355396 20645 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:43:31.355603 20645 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:43:31.357602 20645 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/master-1/data/instance:
uuid: "1d39673cf6e84fd9a7012f26f96107e0"
format_stamp: "Formatted at 2026-09-25 02:43:31 on dist-test-slave-kznw"
server_key: "eb05e85604b765931252c1fe7caccc0c"
server_key_iv: "89e31608de04177c96d975119a6ec95b"
server_key_version: "encryptionkey@0"
I20260925 02:43:31.358129 20645 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/master-1/wal/instance:
uuid: "1d39673cf6e84fd9a7012f26f96107e0"
format_stamp: "Formatted at 2026-09-25 02:43:31 on dist-test-slave-kznw"
server_key: "eb05e85604b765931252c1fe7caccc0c"
server_key_iv: "89e31608de04177c96d975119a6ec95b"
server_key_version: "encryptionkey@0"
I20260925 02:43:31.361749 20645 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260925 02:43:31.364171 20661 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:31.365329 20645 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260925 02:43:31.365447 20645 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/master-1/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/master-1/wal
uuid: "1d39673cf6e84fd9a7012f26f96107e0"
format_stamp: "Formatted at 2026-09-25 02:43:31 on dist-test-slave-kznw"
server_key: "eb05e85604b765931252c1fe7caccc0c"
server_key_iv: "89e31608de04177c96d975119a6ec95b"
server_key_version: "encryptionkey@0"
I20260925 02:43:31.365553 20645 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:43:31.375391 20645 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:43:31.378266 20645 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:43:31.378427 20645 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:43:31.384878 20645 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.189:33117
I20260925 02:43:31.385174 20714 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.189:33117 every 8 connection(s)
I20260925 02:43:31.385795 20645 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/master-1/data/info.pb
I20260925 02:43:31.388502 20715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:43:31.392201 20715 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.157.190" port: 34721 } has no permanent_uuid. Determining permanent_uuid...
I20260925 02:43:31.393582 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 20645
Sep 25 02:43:31 dist-test-slave-kznw krb5kdc[20127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304211, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.157.189@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM
Sep 25 02:43:31 dist-test-slave-kznw krb5kdc[20127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304210, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.13.157.189@KRBTEST.COM
I20260925 02:43:31.405169 20211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:31.394208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42406 (local address 127.13.157.190:34721)
0925 02:43:31.394353 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:31.394358 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:31.395285 (+ 927us) server_negotiation.cc:407] Connection header received
0925 02:43:31.396194 (+ 909us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:31.396198 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:31.396253 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:31.396326 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:31.397126 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:31.397648 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:31.398619 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:31.398855 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:31.398970 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:31.401390 (+ 2420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:31.401394 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:31.401396 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:43:31.401422 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:31.402896 (+ 1474us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:31.403567 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:31.403570 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:31.403571 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:31.403656 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:31.404009 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:31.404012 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:31.404013 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:31.404241 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:31.404327 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:31.404837 (+ 510us) server_negotiation.cc:299] Negotiation successful
0925 02:43:31.404956 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260925 02:43:31.405941 20718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:31.394700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:34721 (local address 127.0.0.1:42406)
0925 02:43:31.395153 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:31.395193 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:31.395996 (+ 803us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:31.396384 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:31.396390 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:31.396736 (+ 346us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:31.396980 (+ 244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:31.396993 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:31.397765 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:31.397767 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:31.398504 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:31.398512 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:31.399006 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:31.399009 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:31.399176 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:31.399721 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:31.399741 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:31.401214 (+ 1473us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:43:31.403024 (+ 1810us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:31.403032 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:31.403090 (+ 58us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:31.403446 (+ 356us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:31.403768 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:31.403772 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:31.403774 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:31.403916 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:31.404344 (+ 428us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:43:31.404351 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:43:31.404625 (+ 274us) client_negotiation.cc:769] Sending connection context
0925 02:43:31.404773 (+ 148us) client_negotiation.cc:240] Negotiation successful
0925 02:43:31.405073 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"mutex_wait_us":1,"thread_start_us":128,"threads_started":1}
I20260925 02:43:31.405946 20719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:31.395455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47540 (local address 127.13.157.189:33117)
0925 02:43:31.395718 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:31.395728 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:31.395760 (+ 32us) server_negotiation.cc:407] Connection header received
0925 02:43:31.396000 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:31.396015 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:31.396233 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:31.396405 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:31.397192 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:31.398049 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:31.398839 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:31.399249 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:31.399340 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:31.401796 (+ 2456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:31.401801 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:31.401811 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:43:31.401831 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:31.403509 (+ 1678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:31.403921 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:31.403926 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:31.403928 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:31.404002 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:31.404253 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:31.404257 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:31.404258 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:31.404623 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:31.404793 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:31.405020 (+ 227us) server_negotiation.cc:299] Negotiation successful
0925 02:43:31.405156 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":32,"server-negotiator.queue_time_us":173,"thread_start_us":74,"threads_started":1}
I20260925 02:43:31.407480 20715 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.157.189" port: 33117 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 02:43:31 dist-test-slave-kznw krb5kdc[20127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304211, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.157.189@KRBTEST.COM for oryx/127.13.157.189@KRBTEST.COM
I20260925 02:43:31.415586 20718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:31.407960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.189:33117 (local address 127.0.0.1:47550)
0925 02:43:31.408066 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:31.408088 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:31.408156 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:31.408462 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:31.408464 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:31.408589 (+ 125us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:31.408676 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:31.408681 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:31.409456 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:31.409459 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:31.409983 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:31.409991 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:31.411007 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:31.411010 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:31.411091 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:31.411515 (+ 424us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:31.411526 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:31.412668 (+ 1142us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:43:31.414226 (+ 1558us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:31.414228 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:31.414229 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:31.414405 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:31.414664 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:31.414666 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:31.414668 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:31.414752 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:31.415182 (+ 430us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:43:31.415184 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:43:31.415258 (+ 74us) client_negotiation.cc:769] Sending connection context
0925 02:43:31.415341 (+ 83us) client_negotiation.cc:240] Negotiation successful
0925 02:43:31.415440 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260925 02:43:31.415619 20719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:31.408174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47550 (local address 127.13.157.189:33117)
0925 02:43:31.408285 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:31.408288 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:31.408298 (+ 10us) server_negotiation.cc:407] Connection header received
0925 02:43:31.408325 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:31.408326 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:31.408358 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:31.408416 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:31.408776 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:31.409358 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:31.410120 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:31.410850 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:31.410973 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:31.412789 (+ 1816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:31.412792 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:31.412794 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:43:31.412813 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:31.414129 (+ 1316us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:31.414487 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:31.414490 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:31.414491 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:31.414562 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:31.414857 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:31.414860 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:31.414861 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:31.415069 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:31.415147 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:31.415383 (+ 236us) server_negotiation.cc:299] Negotiation successful
0925 02:43:31.415493 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":21,"server-negotiator.queue_time_us":43}
I20260925 02:43:31.418433 20715 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d39673cf6e84fd9a7012f26f96107e0: Bootstrap starting.
I20260925 02:43:31.420817 20715 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1d39673cf6e84fd9a7012f26f96107e0: Neither blocks nor log segments found. Creating new log.
I20260925 02:43:31.421644 20715 log.cc:824] T 00000000000000000000000000000000 P 1d39673cf6e84fd9a7012f26f96107e0: Log is configured to *not* fsync() on all Append() calls
I20260925 02:43:31.423785 20715 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d39673cf6e84fd9a7012f26f96107e0: No bootstrap required, opened a new log
I20260925 02:43:31.426651 20715 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1d39673cf6e84fd9a7012f26f96107e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b4937775ce34bed9ca35afdca0243b0" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 34721 } } peers { permanent_uuid: "1d39673cf6e84fd9a7012f26f96107e0" member_type: VOTER last_known_addr { host: "127.13.157.189" port: 33117 } }
I20260925 02:43:31.426839 20715 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1d39673cf6e84fd9a7012f26f96107e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:43:31.426877 20715 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1d39673cf6e84fd9a7012f26f96107e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d39673cf6e84fd9a7012f26f96107e0, State: Initialized, Role: FOLLOWER
I20260925 02:43:31.427315 20715 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1d39673cf6e84fd9a7012f26f96107e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b4937775ce34bed9ca35afdca0243b0" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 34721 } } peers { permanent_uuid: "1d39673cf6e84fd9a7012f26f96107e0" member_type: VOTER last_known_addr { host: "127.13.157.189" port: 33117 } }
I20260925 02:43:31.427755 20721 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d39673cf6e84fd9a7012f26f96107e0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b4937775ce34bed9ca35afdca0243b0" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 34721 } } peers { permanent_uuid: "1d39673cf6e84fd9a7012f26f96107e0" member_type: VOTER last_known_addr { host: "127.13.157.189" port: 33117 } } }
I20260925 02:43:31.427873 20721 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d39673cf6e84fd9a7012f26f96107e0 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 02:43:31.428462 20715 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1d39673cf6e84fd9a7012f26f96107e0 [sys.catalog]: configured and running, proceeding with master startup.
W20260925 02:43:31.432875 20733 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1d39673cf6e84fd9a7012f26f96107e0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 02:43:31.432955 20733 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 02:43:31.434866 20645 master_runner.cc:428] Detected that this master 1d39673cf6e84fd9a7012f26f96107e0 is joining an existing cluster
I20260925 02:43:31.434947 20645 master_runner.cc:432] Initiating AddMaster RPC to add 127.13.157.189:33117
I20260925 02:43:31.445209 20719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:31.437212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47560 (local address 127.13.157.189:33117)
0925 02:43:31.437308 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:31.437312 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:31.437776 (+ 464us) server_negotiation.cc:407] Connection header received
0925 02:43:31.437886 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:31.437888 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:31.437922 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:31.437995 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:31.438571 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:31.439350 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:31.440502 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:31.440723 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:31.440807 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:31.441929 (+ 1122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:31.441933 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:31.441935 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:43:31.441960 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:31.443619 (+ 1659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:31.444041 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:31.444044 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:31.444046 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:31.444123 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:31.444410 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:31.444413 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:31.444415 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:31.444616 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:31.444690 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:31.444949 (+ 259us) server_negotiation.cc:299] Negotiation successful
0925 02:43:31.445053 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260925 02:43:31.445652 20740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:31.437403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.189:33117 (local address 127.0.0.1:47560)
0925 02:43:31.437702 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:31.437715 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:31.437814 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:31.438023 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:31.438026 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:31.438345 (+ 319us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:31.438471 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:31.438479 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:31.439455 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:31.439459 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:31.440395 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:31.440406 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:31.440834 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:31.440838 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:31.440922 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:31.441365 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:31.441378 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:31.441802 (+ 424us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:43:31.443721 (+ 1919us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:31.443724 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:31.443726 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:31.443941 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:31.444220 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:31.444223 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:31.444225 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:31.444322 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:31.444697 (+ 375us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:43:31.444700 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:43:31.444790 (+ 90us) client_negotiation.cc:769] Sending connection context
0925 02:43:31.445324 (+ 534us) client_negotiation.cc:240] Negotiation successful
0925 02:43:31.445480 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233,"mutex_wait_us":24,"thread_start_us":88,"threads_started":1}
I20260925 02:43:31.446828 20211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:31.437204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42410 (local address 127.13.157.190:34721)
0925 02:43:31.437317 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:31.437321 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:31.437596 (+ 275us) server_negotiation.cc:407] Connection header received
0925 02:43:31.437756 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:31.437759 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:31.437809 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:31.437878 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:31.438546 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:31.439350 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:31.441606 (+ 2256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:31.441926 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:31.442012 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:31.443609 (+ 1597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:31.443612 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:31.443615 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:43:31.443639 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:31.445165 (+ 1526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:31.445613 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:31.445617 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:31.445619 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:31.445695 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:31.446106 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:31.446109 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:31.446111 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:31.446279 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:31.446333 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:31.446620 (+ 287us) server_negotiation.cc:299] Negotiation successful
0925 02:43:31.446722 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260925 02:43:31.447237 20739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:31.437129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:34721 (local address 127.0.0.1:42410)
0925 02:43:31.437492 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:31.437506 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:31.437664 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:31.438074 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:31.438076 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:31.438324 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:31.438440 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:31.438445 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:31.440539 (+ 2094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:31.440542 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:31.441434 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:31.441444 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:31.442021 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:31.442024 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:31.442185 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:31.443067 (+ 882us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:31.443077 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:31.443484 (+ 407us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:43:31.445260 (+ 1776us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:31.445263 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:31.445265 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:31.445528 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:31.445787 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:31.445790 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:31.445792 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:31.445992 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:31.446388 (+ 396us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:43:31.446390 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:43:31.446455 (+ 65us) client_negotiation.cc:769] Sending connection context
0925 02:43:31.446978 (+ 523us) client_negotiation.cc:240] Negotiation successful
0925 02:43:31.447117 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"spinlock_wait_cycles":8960,"thread_start_us":81,"threads_started":1}
Sep 25 02:43:31 dist-test-slave-kznw krb5kdc[20127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304210, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.157.190@KRBTEST.COM for oryx/127.13.157.189@KRBTEST.COM
I20260925 02:43:31.458412 20719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:31.450901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47574 (local address 127.13.157.189:33117)
0925 02:43:31.451010 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:31.451014 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:31.451309 (+ 295us) server_negotiation.cc:407] Connection header received
0925 02:43:31.451474 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:31.451477 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:31.451522 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:31.451586 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:31.452154 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:31.452607 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:31.453402 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:31.453615 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:31.453693 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:31.455668 (+ 1975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:31.455671 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:31.455673 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:43:31.455692 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:31.456886 (+ 1194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:31.457289 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:31.457292 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:31.457294 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:31.457361 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:31.457601 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:31.457605 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:31.457606 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:31.457755 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:31.457835 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:31.458179 (+ 344us) server_negotiation.cc:299] Negotiation successful
0925 02:43:31.458266 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260925 02:43:31.458432 20741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:31.450881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.189:33117 (local address 127.0.0.1:47574)
0925 02:43:31.451201 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:31.451242 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:31.451363 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:31.451633 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:31.451640 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:31.451953 (+ 313us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:31.452050 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:31.452056 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:31.452699 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:31.452703 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:31.453300 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:31.453308 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:31.453705 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:31.453709 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:31.453807 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:31.454233 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:31.454243 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:31.455548 (+ 1305us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:43:31.456982 (+ 1434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:31.456991 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:31.456997 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:31.457204 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:31.457438 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:31.457441 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:31.457443 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:31.457518 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:31.457852 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:43:31.457858 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:43:31.457963 (+ 105us) client_negotiation.cc:769] Sending connection context
0925 02:43:31.458081 (+ 118us) client_negotiation.cc:240] Negotiation successful
0925 02:43:31.458255 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":80,"threads_started":1}
I20260925 02:43:31.459941 20171 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.13.157.189:33117: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "1d39673cf6e84fd9a7012f26f96107e0"
member_type: NON_VOTER
last_known_addr {
host: "127.13.157.189"
port: 33117
}
attrs {
promote: true
}
}
dest_uuid: "2b4937775ce34bed9ca35afdca0243b0"
cas_config_opid_index: -1
I20260925 02:43:31.460644 20171 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2b4937775ce34bed9ca35afdca0243b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "2b4937775ce34bed9ca35afdca0243b0" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 34721 } } peers { permanent_uuid: "1d39673cf6e84fd9a7012f26f96107e0" member_type: NON_VOTER last_known_addr { host: "127.13.157.189" port: 33117 } attrs { promote: true } }
I20260925 02:43:31.462368 20744 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 2b4937775ce34bed9ca35afdca0243b0 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 1d39673cf6e84fd9a7012f26f96107e0 (127.13.157.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "2b4937775ce34bed9ca35afdca0243b0" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 34721 } } peers { permanent_uuid: "1d39673cf6e84fd9a7012f26f96107e0" member_type: NON_VOTER last_known_addr { host: "127.13.157.189" port: 33117 } attrs { promote: true } } }
I20260925 02:43:31.463164 20689 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1d39673cf6e84fd9a7012f26f96107e0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:43:31.464361 20744 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.13.157.189:33117
I20260925 02:43:31.464494 20689 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1d39673cf6e84fd9a7012f26f96107e0 [term 1 FOLLOWER]: Refusing update from remote peer 2b4937775ce34bed9ca35afdca0243b0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260925 02:43:31.464778 20645 master.cc:537] Master@127.13.157.189:33117 shutting down...
I20260925 02:43:31.465183 20744 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b4937775ce34bed9ca35afdca0243b0 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "2b4937775ce34bed9ca35afdca0243b0" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "2b4937775ce34bed9ca35afdca0243b0" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 34721 } } peers { permanent_uuid: "1d39673cf6e84fd9a7012f26f96107e0" member_type: NON_VOTER last_known_addr { host: "127.13.157.189" port: 33117 } attrs { promote: true } } }
I20260925 02:43:31.465281 20744 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b4937775ce34bed9ca35afdca0243b0 [sys.catalog]: This master's current role is: LEADER
I20260925 02:43:31.465442 20744 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2b4937775ce34bed9ca35afdca0243b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d39673cf6e84fd9a7012f26f96107e0" member_type: NON_VOTER last_known_addr { host: "127.13.157.189" port: 33117 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260925 02:43:31.467511 20159 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
WARNING: no policy specified for oryx/127.13.157.188@KRBTEST.COM; defaulting to no policy
W20260925 02:43:31.468551 20159 consensus_peers.cc:487] T 00000000000000000000000000000000 P 2b4937775ce34bed9ca35afdca0243b0 -> Peer 1d39673cf6e84fd9a7012f26f96107e0 (127.13.157.189:33117): Couldn't send request to peer 1d39673cf6e84fd9a7012f26f96107e0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260925 02:43:31.469300 20645 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 1d39673cf6e84fd9a7012f26f96107e0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 02:43:31.469556 20645 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 1d39673cf6e84fd9a7012f26f96107e0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:43:31.469667 20645 tablet_replica.cc:330] T 00000000000000000000000000000000 P 1d39673cf6e84fd9a7012f26f96107e0: stopping tablet replica
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.157.188@KRBTEST.COM" created.
I20260925 02:43:31.475167 20645 master.cc:552] Master@127.13.157.189:33117 shutdown complete.
I20260925 02:43:31.475261 20645 master_runner.cc:305] Clearing existing system tablet
I20260925 02:43:31.475533 20645 fs_manager.cc:359] Metadata directory not provided
I20260925 02:43:31.475622 20645 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:43:31.476794 20645 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 02:43:31.477592 20750 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:31.477804 20645 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:43:31.477909 20645 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 02:43:31.477939 20645 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/master-1/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/master-1/wal
uuid: "1d39673cf6e84fd9a7012f26f96107e0"
format_stamp: "Formatted at 2026-09-25 02:43:31 on dist-test-slave-kznw"
server_key: "eb05e85604b765931252c1fe7caccc0c"
server_key_iv: "89e31608de04177c96d975119a6ec95b"
server_key_version: "encryptionkey@0"
I20260925 02:43:31.479949 20645 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 1d39673cf6e84fd9a7012f26f96107e0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:43:31.482357 20645 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 1d39673cf6e84fd9a7012f26f96107e0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260925 02:43:31.482443 20645 log.cc:1196] T 00000000000000000000000000000000 P 1d39673cf6e84fd9a7012f26f96107e0: Deleting WAL directory at /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260925 02:43:31.482681 20645 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 1d39673cf6e84fd9a7012f26f96107e0: Deleting consensus metadata
I20260925 02:43:31.482872 20645 master_runner.cc:315] Copying system tablet from 127.13.157.190:34721
I20260925 02:43:31.484057 20645 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 1d39673cf6e84fd9a7012f26f96107e0: tablet copy: Beginning tablet copy session from remote peer at address 127.13.157.190:34721
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.157.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/krb5kdc/oryx_127.13.157.188.keytab.
Entry for principal oryx/127.13.157.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/krb5kdc/oryx_127.13.157.188.keytab.
I20260925 02:43:31.489339 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.157.188:46853
--webserver_interface=127.13.157.188
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/krb5kdc/oryx_127.13.157.188.keytab
--principal=oryx/127.13.157.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:46123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.157.190:34721,127.13.157.189:33117,127.13.157.188:46853
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 02:43:31.494365 20757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:31.484595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:34721 (local address 127.0.0.1:42426)
0925 02:43:31.484919 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:31.484933 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:31.485043 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:31.485284 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:31.485288 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:31.485480 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:31.485729 (+ 249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:31.485737 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:31.486540 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:31.486544 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:31.487445 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:31.487456 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:31.487908 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:31.487912 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:31.487996 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:31.488464 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:31.488478 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:31.490814 (+ 2336us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:43:31.492745 (+ 1931us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:31.492749 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:31.492751 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:31.492990 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:31.493313 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:31.493316 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:31.493318 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:31.493405 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:31.493828 (+ 423us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:43:31.493831 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:43:31.493930 (+ 99us) client_negotiation.cc:769] Sending connection context
0925 02:43:31.494029 (+ 99us) client_negotiation.cc:240] Negotiation successful
0925 02:43:31.494163 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":81,"threads_started":1}
I20260925 02:43:31.494439 20211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:31.484668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42426 (local address 127.13.157.190:34721)
0925 02:43:31.484777 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:31.484781 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:31.485006 (+ 225us) server_negotiation.cc:407] Connection header received
0925 02:43:31.485126 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:31.485128 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:31.485177 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:31.485253 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:31.485856 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:31.486428 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:31.487571 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:31.487803 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:31.487885 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:31.491000 (+ 3115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:31.491005 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:31.491007 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:43:31.491084 (+ 77us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:31.492646 (+ 1562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:31.493122 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:31.493126 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:31.493128 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:31.493221 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:31.493488 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:31.493491 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:31.493493 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:31.493736 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:31.493831 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:31.494105 (+ 274us) server_negotiation.cc:299] Negotiation successful
0925 02:43:31.494285 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260925 02:43:31.495348 20192 tablet_copy_service.cc:140] P 2b4937775ce34bed9ca35afdca0243b0: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 1d39673cf6e84fd9a7012f26f96107e0 ({username='oryx', principal='oryx/127.13.157.189@KRBTEST.COM'} at 127.0.0.1:42426)
I20260925 02:43:31.495535 20192 tablet_copy_service.cc:161] P 2b4937775ce34bed9ca35afdca0243b0: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 1d39673cf6e84fd9a7012f26f96107e0 at {username='oryx', principal='oryx/127.13.157.189@KRBTEST.COM'} at 127.0.0.1:42426: session id = 1d39673cf6e84fd9a7012f26f96107e0-00000000000000000000000000000000
I20260925 02:43:31.497253 20192 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 2b4937775ce34bed9ca35afdca0243b0: Tablet Copy: opened 0 blocks and 1 log segments
I20260925 02:43:31.499049 20645 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260925 02:43:31.501428 20645 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 1d39673cf6e84fd9a7012f26f96107e0: tablet copy: Starting download of 0 data blocks...
I20260925 02:43:31.501627 20645 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 1d39673cf6e84fd9a7012f26f96107e0: tablet copy: Starting download of 1 WAL segments...
I20260925 02:43:31.503427 20645 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 1d39673cf6e84fd9a7012f26f96107e0: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260925 02:43:31.505231 20192 tablet_copy_service.cc:342] P 2b4937775ce34bed9ca35afdca0243b0: Request end of tablet copy session 1d39673cf6e84fd9a7012f26f96107e0-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.13.157.189@KRBTEST.COM'} at 127.0.0.1:42426
I20260925 02:43:31.505357 20192 tablet_copy_service.cc:433] P 2b4937775ce34bed9ca35afdca0243b0: ending tablet copy session 1d39673cf6e84fd9a7012f26f96107e0-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 1d39673cf6e84fd9a7012f26f96107e0
I20260925 02:43:31.512849 20645 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:43:31.514532 20762 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:43:31.514796 20645 server_base.cc:1034] running on GCE node
W20260925 02:43:31.514765 20759 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:31.515012 20760 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:43:31.515311 20645 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:43:31.515604 20645 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:43:31.516736 20645 hybrid_clock.cc:648] HybridClock initialized: now 1790304211516722 us; error 30 us; skew 500 ppm
Sep 25 02:43:31 dist-test-slave-kznw krb5kdc[20127](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304211, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.157.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:43:31.518186 20645 init.cc:377] Logged in from keytab as oryx/127.13.157.189@KRBTEST.COM (short username oryx)
I20260925 02:43:31.518625 20645 webserver.cc:460] Webserver started at http://127.13.157.189:36037/ using document root <none> and password file <none>
I20260925 02:43:31.518800 20645 fs_manager.cc:359] Metadata directory not provided
I20260925 02:43:31.518838 20645 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:43:31.520210 20645 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 02:43:31.521152 20769 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:31.521410 20645 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 02:43:31.521483 20645 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/master-1/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/master-1/wal
uuid: "1d39673cf6e84fd9a7012f26f96107e0"
format_stamp: "Formatted at 2026-09-25 02:43:31 on dist-test-slave-kznw"
server_key: "eb05e85604b765931252c1fe7caccc0c"
server_key_iv: "89e31608de04177c96d975119a6ec95b"
server_key_version: "encryptionkey@0"
I20260925 02:43:31.521569 20645 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:43:31.530331 20645 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:43:31.532236 20645 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:43:31.536872 20645 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.189:33117
I20260925 02:43:31.537506 20645 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/master-1/data/info.pb
I20260925 02:43:31.539047 20823 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.189:33117 every 8 connection(s)
I20260925 02:43:31.539335 20824 sys_catalog.cc:263] Verifying existing consensus state
I20260925 02:43:31.540467 20824 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d39673cf6e84fd9a7012f26f96107e0: Bootstrap starting.
I20260925 02:43:31.550752 20824 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d39673cf6e84fd9a7012f26f96107e0: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 02:43:31.551090 20824 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d39673cf6e84fd9a7012f26f96107e0: Bootstrap complete.
I20260925 02:43:31.551683 20824 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1d39673cf6e84fd9a7012f26f96107e0 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "2b4937775ce34bed9ca35afdca0243b0" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 34721 } } peers { permanent_uuid: "1d39673cf6e84fd9a7012f26f96107e0" member_type: NON_VOTER last_known_addr { host: "127.13.157.189" port: 33117 } attrs { promote: true } }
I20260925 02:43:31.551823 20824 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1d39673cf6e84fd9a7012f26f96107e0 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 1d39673cf6e84fd9a7012f26f96107e0, State: Initialized, Role: LEARNER
I20260925 02:43:31.551929 20824 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1d39673cf6e84fd9a7012f26f96107e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "2b4937775ce34bed9ca35afdca0243b0" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 34721 } } peers { permanent_uuid: "1d39673cf6e84fd9a7012f26f96107e0" member_type: NON_VOTER last_known_addr { host: "127.13.157.189" port: 33117 } attrs { promote: true } }
I20260925 02:43:31.552261 20827 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d39673cf6e84fd9a7012f26f96107e0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "2b4937775ce34bed9ca35afdca0243b0" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 34721 } } peers { permanent_uuid: "1d39673cf6e84fd9a7012f26f96107e0" member_type: NON_VOTER last_known_addr { host: "127.13.157.189" port: 33117 } attrs { promote: true } } }
I20260925 02:43:31.552392 20824 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1d39673cf6e84fd9a7012f26f96107e0 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:43:31.552457 20827 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d39673cf6e84fd9a7012f26f96107e0 [sys.catalog]: This master's current role is: LEARNER
I20260925 02:43:31.555958 20838 catalog_manager.cc:1259] Loaded cluster ID: 19dd28aa2b484a96bcdd464cb2d07319
I20260925 02:43:31.556017 20838 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1d39673cf6e84fd9a7012f26f96107e0: loading cluster ID for follower catalog manager: success
I20260925 02:43:31.558005 20838 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1d39673cf6e84fd9a7012f26f96107e0: acquiring CA information for follower catalog manager: success
I20260925 02:43:31.559381 20838 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1d39673cf6e84fd9a7012f26f96107e0: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260925 02:43:31.612715 20758 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:43:31.612995 20758 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:43:31.613036 20758 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:43:31.617292 20758 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:43:31.617362 20758 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:43:31.617383 20758 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:43:31.617398 20758 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:43:31.617415 20758 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:43:31.623229 20758 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:46123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.13.157.190:34721,127.13.157.189:33117,127.13.157.188:46853
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/krb5kdc/oryx_127.13.157.188.keytab
--principal=oryx/127.13.157.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.157.188:46853
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.13.157.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20758
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:43:31.624408 20758 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:43:31.625531 20758 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:43:31.633767 20844 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:31.633776 20843 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:43:31.633852 20758 server_base.cc:1034] running on GCE node
W20260925 02:43:31.633767 20846 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:43:31.634649 20758 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:43:31.635815 20758 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:43:31.636991 20758 hybrid_clock.cc:648] HybridClock initialized: now 1790304211636968 us; error 38 us; skew 500 ppm
Sep 25 02:43:31 dist-test-slave-kznw krb5kdc[20127](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304211, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.157.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:43:31.639775 20758 init.cc:377] Logged in from keytab as oryx/127.13.157.188@KRBTEST.COM (short username oryx)
I20260925 02:43:31.640689 20758 webserver.cc:460] Webserver started at http://127.13.157.188:40659/ using document root <none> and password file <none>
I20260925 02:43:31.641219 20758 fs_manager.cc:359] Metadata directory not provided
I20260925 02:43:31.641291 20758 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:43:31.641531 20758 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:43:31.643724 20758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/master-2/data/instance:
uuid: "82186af4c8cb4282a83c2d2936d226ac"
format_stamp: "Formatted at 2026-09-25 02:43:31 on dist-test-slave-kznw"
server_key: "b00244ae50c3e58ac66c51f1ca23940d"
server_key_iv: "077ff01cbe6f08d79a9c15d62d0029e1"
server_key_version: "encryptionkey@0"
I20260925 02:43:31.644289 20758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/master-2/wal/instance:
uuid: "82186af4c8cb4282a83c2d2936d226ac"
format_stamp: "Formatted at 2026-09-25 02:43:31 on dist-test-slave-kznw"
server_key: "b00244ae50c3e58ac66c51f1ca23940d"
server_key_iv: "077ff01cbe6f08d79a9c15d62d0029e1"
server_key_version: "encryptionkey@0"
I20260925 02:43:31.648123 20758 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260925 02:43:31.650633 20853 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:31.652153 20758 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 02:43:31.652278 20758 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/master-2/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/master-2/wal
uuid: "82186af4c8cb4282a83c2d2936d226ac"
format_stamp: "Formatted at 2026-09-25 02:43:31 on dist-test-slave-kznw"
server_key: "b00244ae50c3e58ac66c51f1ca23940d"
server_key_iv: "077ff01cbe6f08d79a9c15d62d0029e1"
server_key_version: "encryptionkey@0"
I20260925 02:43:31.652390 20758 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:43:31.685774 20758 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:43:31.688946 20758 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:43:31.689113 20758 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:43:31.696264 20758 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.188:46853
I20260925 02:43:31.696589 20905 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.188:46853 every 8 connection(s)
I20260925 02:43:31.697307 20758 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/master-2/data/info.pb
I20260925 02:43:31.700145 20906 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:43:31.703889 20906 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.157.190" port: 34721 } has no permanent_uuid. Determining permanent_uuid...
I20260925 02:43:31.706333 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 20758
Sep 25 02:43:31 dist-test-slave-kznw krb5kdc[20127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304211, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.157.188@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM
Sep 25 02:43:31 dist-test-slave-kznw krb5kdc[20127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304210, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.13.157.188@KRBTEST.COM
I20260925 02:43:31.717434 20211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:31.705807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32814 (local address 127.13.157.190:34721)
0925 02:43:31.705948 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:31.705952 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:31.706754 (+ 802us) server_negotiation.cc:407] Connection header received
0925 02:43:31.707994 (+ 1240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:31.707997 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:31.708045 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:31.708130 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:31.709023 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:31.709566 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:31.710652 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:31.710880 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:31.710987 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:31.713554 (+ 2567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:31.713558 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:31.713561 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:43:31.713612 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:31.715132 (+ 1520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:31.715653 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:31.715657 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:31.715659 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:31.715748 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:31.716387 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:31.716390 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:31.716391 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:31.716589 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:31.716684 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:31.717158 (+ 474us) server_negotiation.cc:299] Negotiation successful
0925 02:43:31.717273 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260925 02:43:31.718259 20908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:31.706093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:34721 (local address 127.0.0.1:32814)
0925 02:43:31.706601 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:31.706656 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:31.707768 (+ 1112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:31.708189 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:31.708195 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:31.708625 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:31.708860 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:31.708876 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:31.709692 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:31.709696 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:31.710540 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:31.710549 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:31.711044 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:31.711048 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:31.711260 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:31.711861 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:31.711882 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:31.713399 (+ 1517us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:43:31.715225 (+ 1826us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:31.715233 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:31.715240 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:31.715564 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:31.715836 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:31.715839 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:31.715841 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:31.716011 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:31.716692 (+ 681us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:43:31.716697 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:43:31.716927 (+ 230us) client_negotiation.cc:769] Sending connection context
0925 02:43:31.717088 (+ 161us) client_negotiation.cc:240] Negotiation successful
0925 02:43:31.717390 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"mutex_wait_us":52,"thread_start_us":132,"threads_started":1}
I20260925 02:43:31.718271 20909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:31.707170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34792 (local address 127.13.157.188:46853)
0925 02:43:31.707432 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:31.707442 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:31.707482 (+ 40us) server_negotiation.cc:407] Connection header received
0925 02:43:31.707775 (+ 293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:31.707797 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:31.708026 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:31.708186 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:31.708921 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:31.709852 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:31.710686 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:31.711198 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:31.711293 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:31.713984 (+ 2691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:31.713991 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:31.714002 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:43:31.714028 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:31.715617 (+ 1589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:31.716129 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:31.716134 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:31.716136 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:31.716351 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:31.716632 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:31.716636 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:31.716637 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:31.716919 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:31.717067 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:31.717310 (+ 243us) server_negotiation.cc:299] Negotiation successful
0925 02:43:31.717438 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":35,"server-negotiator.queue_time_us":174,"thread_start_us":77,"threads_started":1}
I20260925 02:43:31.719682 20906 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.157.189" port: 33117 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 02:43:31 dist-test-slave-kznw krb5kdc[20127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304211, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.157.188@KRBTEST.COM for oryx/127.13.157.189@KRBTEST.COM
I20260925 02:43:31.727048 20908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:31.720169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.189:33117 (local address 127.0.0.1:56730)
0925 02:43:31.720312 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:31.720325 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:31.720389 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:31.720778 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:31.720780 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:31.720898 (+ 118us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:31.720985 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:31.720990 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:31.721829 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:31.721831 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:31.722370 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:31.722378 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:31.722802 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:31.722807 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:31.722899 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:31.723345 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:31.723355 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:31.724460 (+ 1105us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:43:31.725949 (+ 1489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:31.725951 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:31.725953 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:31.726111 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:31.726323 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:31.726325 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:31.726326 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:31.726390 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:31.726649 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:43:31.726651 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:43:31.726718 (+ 67us) client_negotiation.cc:769] Sending connection context
0925 02:43:31.726788 (+ 70us) client_negotiation.cc:240] Negotiation successful
0925 02:43:31.726873 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":83}
I20260925 02:43:31.727128 20910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:31.720263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56730 (local address 127.13.157.189:33117)
0925 02:43:31.720588 (+ 325us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:31.720592 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:31.720602 (+ 10us) server_negotiation.cc:407] Connection header received
0925 02:43:31.720635 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:31.720637 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:31.720675 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:31.720741 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:31.721072 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:31.721746 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:31.722473 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:31.722710 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:31.722808 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:31.724583 (+ 1775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:31.724586 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:31.724588 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:43:31.724610 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:31.725871 (+ 1261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:31.726178 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:31.726182 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:31.726183 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:31.726257 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:31.726441 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:31.726443 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:31.726444 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:31.726589 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:31.726653 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:31.726821 (+ 168us) server_negotiation.cc:299] Negotiation successful
0925 02:43:31.726930 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":259,"thread_start_us":111,"threads_started":1}
I20260925 02:43:31.727844 20906 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.157.188" port: 46853 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 02:43:31 dist-test-slave-kznw krb5kdc[20127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304211, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.157.188@KRBTEST.COM for oryx/127.13.157.188@KRBTEST.COM
I20260925 02:43:31.734375 20908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:31.728257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.188:46853 (local address 127.0.0.1:34804)
0925 02:43:31.728360 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:31.728372 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:31.728433 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:31.728599 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:31.728601 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:31.728710 (+ 109us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:31.728770 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:31.728774 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:31.729377 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:31.729379 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:31.729863 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:31.729869 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:31.730225 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:31.730227 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:31.730280 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:31.730616 (+ 336us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:31.730627 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:31.731782 (+ 1155us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:43:31.733159 (+ 1377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:31.733162 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:31.733164 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:31.733328 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:31.733549 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:31.733552 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:31.733554 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:31.733629 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:31.733927 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:43:31.733930 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:43:31.734005 (+ 75us) client_negotiation.cc:769] Sending connection context
0925 02:43:31.734091 (+ 86us) client_negotiation.cc:240] Negotiation successful
0925 02:43:31.734207 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260925 02:43:31.734383 20909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:31.728309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34804 (local address 127.13.157.188:46853)
0925 02:43:31.728406 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:31.728410 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:31.728422 (+ 12us) server_negotiation.cc:407] Connection header received
0925 02:43:31.728487 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:31.728489 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:31.728531 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:31.728588 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:31.728840 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:31.729303 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:31.729936 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:31.730149 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:31.730213 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:31.731895 (+ 1682us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:31.731898 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:31.731900 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:43:31.731919 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:31.733066 (+ 1147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:31.733404 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:31.733407 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:31.733409 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:31.733472 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:31.733701 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:31.733704 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:31.733705 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:31.733848 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:31.733927 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:31.734144 (+ 217us) server_negotiation.cc:299] Negotiation successful
0925 02:43:31.734260 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":25,"server-negotiator.queue_time_us":34}
I20260925 02:43:31.737342 20906 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82186af4c8cb4282a83c2d2936d226ac: Bootstrap starting.
I20260925 02:43:31.739629 20906 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 82186af4c8cb4282a83c2d2936d226ac: Neither blocks nor log segments found. Creating new log.
I20260925 02:43:31.740314 20906 log.cc:824] T 00000000000000000000000000000000 P 82186af4c8cb4282a83c2d2936d226ac: Log is configured to *not* fsync() on all Append() calls
I20260925 02:43:31.742414 20906 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82186af4c8cb4282a83c2d2936d226ac: No bootstrap required, opened a new log
I20260925 02:43:31.746037 20906 raft_consensus.cc:348] T 00000000000000000000000000000000 P 82186af4c8cb4282a83c2d2936d226ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b4937775ce34bed9ca35afdca0243b0" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 34721 } } peers { permanent_uuid: "1d39673cf6e84fd9a7012f26f96107e0" member_type: VOTER last_known_addr { host: "127.13.157.189" port: 33117 } } peers { permanent_uuid: "82186af4c8cb4282a83c2d2936d226ac" member_type: VOTER last_known_addr { host: "127.13.157.188" port: 46853 } }
I20260925 02:43:31.746304 20906 raft_consensus.cc:374] T 00000000000000000000000000000000 P 82186af4c8cb4282a83c2d2936d226ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:43:31.746362 20906 raft_consensus.cc:729] T 00000000000000000000000000000000 P 82186af4c8cb4282a83c2d2936d226ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82186af4c8cb4282a83c2d2936d226ac, State: Initialized, Role: FOLLOWER
I20260925 02:43:31.746853 20906 consensus_queue.cc:260] T 00000000000000000000000000000000 P 82186af4c8cb4282a83c2d2936d226ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b4937775ce34bed9ca35afdca0243b0" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 34721 } } peers { permanent_uuid: "1d39673cf6e84fd9a7012f26f96107e0" member_type: VOTER last_known_addr { host: "127.13.157.189" port: 33117 } } peers { permanent_uuid: "82186af4c8cb4282a83c2d2936d226ac" member_type: VOTER last_known_addr { host: "127.13.157.188" port: 46853 } }
I20260925 02:43:31.747375 20912 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82186af4c8cb4282a83c2d2936d226ac [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b4937775ce34bed9ca35afdca0243b0" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 34721 } } peers { permanent_uuid: "1d39673cf6e84fd9a7012f26f96107e0" member_type: VOTER last_known_addr { host: "127.13.157.189" port: 33117 } } peers { permanent_uuid: "82186af4c8cb4282a83c2d2936d226ac" member_type: VOTER last_known_addr { host: "127.13.157.188" port: 46853 } } }
I20260925 02:43:31.747491 20912 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82186af4c8cb4282a83c2d2936d226ac [sys.catalog]: This master's current role is: FOLLOWER
I20260925 02:43:31.748270 20906 sys_catalog.cc:564] T 00000000000000000000000000000000 P 82186af4c8cb4282a83c2d2936d226ac [sys.catalog]: configured and running, proceeding with master startup.
W20260925 02:43:31.752817 20924 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 82186af4c8cb4282a83c2d2936d226ac: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 02:43:31.752894 20924 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 02:43:31.756062 20758 master_runner.cc:428] Detected that this master 82186af4c8cb4282a83c2d2936d226ac is joining an existing cluster
I20260925 02:43:31.756114 20758 master_runner.cc:432] Initiating AddMaster RPC to add 127.13.157.188:46853
I20260925 02:43:31.766171 20930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:31.758129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:34721 (local address 127.0.0.1:32820)
0925 02:43:31.758461 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:31.758476 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:31.758615 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:31.759013 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:31.759016 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:31.759243 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:31.759389 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:31.759396 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:31.760467 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:31.760470 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:31.761447 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:31.761457 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:31.762125 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:31.762129 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:31.762209 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:31.762760 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:31.762773 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:31.763173 (+ 400us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:43:31.764918 (+ 1745us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:31.764921 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:31.764922 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:31.765126 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:31.765362 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:31.765363 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:31.765365 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:31.765430 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:31.765779 (+ 349us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:43:31.765781 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:43:31.765860 (+ 79us) client_negotiation.cc:769] Sending connection context
0925 02:43:31.765932 (+ 72us) client_negotiation.cc:240] Negotiation successful
0925 02:43:31.766028 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"mutex_wait_us":1,"thread_start_us":89,"threads_started":1}
I20260925 02:43:31.766269 20211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:31.758180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32820 (local address 127.13.157.190:34721)
0925 02:43:31.758303 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:31.758307 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:31.758534 (+ 227us) server_negotiation.cc:407] Connection header received
0925 02:43:31.758730 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:31.758734 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:31.758787 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:31.758839 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:31.759510 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:31.760343 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:31.761584 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:31.761905 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:31.761970 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:31.763301 (+ 1331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:31.763305 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:31.763308 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:43:31.763334 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:31.764811 (+ 1477us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:31.765203 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:31.765206 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:31.765208 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:31.765290 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:31.765496 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:31.765498 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:31.765500 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:31.765704 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:31.765797 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:31.765995 (+ 198us) server_negotiation.cc:299] Negotiation successful
0925 02:43:31.766116 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260925 02:43:31.768635 20366 heartbeater.cc:503] Master 127.13.157.190:34721 was elected leader, sending a full tablet report...
I20260925 02:43:31.771518 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 20234
Sep 25 02:43:31 dist-test-slave-kznw krb5kdc[20127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304210, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.157.190@KRBTEST.COM for oryx/127.13.157.188@KRBTEST.COM
I20260925 02:43:31.778344 20741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:31.769565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.188:46853 (local address 127.0.0.1:34822)
0925 02:43:31.769652 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:31.769665 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:31.769729 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:31.770328 (+ 599us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:31.770330 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:31.770491 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:31.770581 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:31.770586 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:31.771495 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:31.771499 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:31.772225 (+ 726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:31.772235 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:31.772860 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:31.772863 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:31.772964 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:31.773413 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:31.773427 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:31.774938 (+ 1511us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:43:31.776749 (+ 1811us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:31.776753 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:31.776755 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:31.776982 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:31.777359 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:31.777362 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:31.777364 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:31.777447 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:31.777858 (+ 411us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:43:31.777861 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:43:31.777952 (+ 91us) client_negotiation.cc:769] Sending connection context
0925 02:43:31.778038 (+ 86us) client_negotiation.cc:240] Negotiation successful
0925 02:43:31.778174 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260925 02:43:31.778867 20933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:31.769792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34822 (local address 127.13.157.188:46853)
0925 02:43:31.770072 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:31.770079 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:31.770092 (+ 13us) server_negotiation.cc:407] Connection header received
0925 02:43:31.770152 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:31.770155 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:31.770200 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:31.770281 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:31.770700 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:31.771387 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:31.772354 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:31.772728 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:31.773172 (+ 444us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:31.775106 (+ 1934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:31.775110 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:31.775113 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:43:31.775137 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:31.776630 (+ 1493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:31.777181 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:31.777184 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:31.777186 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:31.777264 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:31.777545 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:31.777548 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:31.777550 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:31.777753 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:31.778512 (+ 759us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:31.778586 (+ 74us) server_negotiation.cc:299] Negotiation successful
0925 02:43:31.778697 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":202,"thread_start_us":110,"threads_started":1}
I20260925 02:43:31.780035 20171 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.13.157.188:46853: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "82186af4c8cb4282a83c2d2936d226ac"
member_type: NON_VOTER
last_known_addr {
host: "127.13.157.188"
port: 46853
}
attrs {
promote: true
}
}
dest_uuid: "2b4937775ce34bed9ca35afdca0243b0"
cas_config_opid_index: 5
I20260925 02:43:31.780357 20931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:31.758425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.189:33117 (local address 127.0.0.1:56746)
0925 02:43:31.758774 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:31.758788 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:31.758954 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:31.759313 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:31.759316 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:31.759545 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:31.759663 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:31.759671 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:31.760761 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:31.760764 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:31.761704 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:31.761714 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:31.762432 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:31.762435 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:31.762596 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:31.775673 (+ 13077us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:31.775687 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:31.776355 (+ 668us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:43:31.778183 (+ 1828us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:31.778186 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:31.778188 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:31.778399 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:31.778816 (+ 417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:31.778819 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:31.778820 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:31.778985 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:31.779874 (+ 889us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:43:31.779876 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:43:31.779962 (+ 86us) client_negotiation.cc:769] Sending connection context
0925 02:43:31.780063 (+ 101us) client_negotiation.cc:240] Negotiation successful
0925 02:43:31.780188 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"mutex_wait_us":69,"thread_start_us":101,"threads_started":1}
I20260925 02:43:31.780391 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.157.129:46073
--local_ip_for_outbound_sockets=127.13.157.129
--tserver_master_addrs=127.13.157.190:34721
--webserver_port=39687
--webserver_interface=127.13.157.129
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/krb5kdc/oryx_127.13.157.129.keytab
--principal=oryx/127.13.157.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:46123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.13.157.190:34721,127.13.157.189:33117
--tserver_master_addrs=127.13.157.190:34721,127.13.157.189:33117,127.13.157.188:46853 with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 02:43:31.780659 20171 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2b4937775ce34bed9ca35afdca0243b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "2b4937775ce34bed9ca35afdca0243b0" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 34721 } } peers { permanent_uuid: "1d39673cf6e84fd9a7012f26f96107e0" member_type: NON_VOTER last_known_addr { host: "127.13.157.189" port: 33117 } attrs { promote: true } } peers { permanent_uuid: "82186af4c8cb4282a83c2d2936d226ac" member_type: NON_VOTER last_known_addr { host: "127.13.157.188" port: 46853 } attrs { promote: true } }
I20260925 02:43:31.780900 20910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:31.758220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56746 (local address 127.13.157.189:33117)
0925 02:43:31.758337 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:31.758341 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:31.758866 (+ 525us) server_negotiation.cc:407] Connection header received
0925 02:43:31.759078 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:31.759081 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:31.759133 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:31.759180 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:31.759820 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:31.760626 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:31.761848 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:31.762212 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:31.762283 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:31.776525 (+ 14242us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:31.776529 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:31.776531 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:43:31.776563 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:31.778065 (+ 1502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:31.778622 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:31.778626 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:31.778628 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:31.778720 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:31.779104 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:31.779108 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:31.779109 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:31.779325 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:31.779417 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:31.780634 (+ 1217us) server_negotiation.cc:299] Negotiation successful
0925 02:43:31.780759 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260925 02:43:31.782164 20880 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 82186af4c8cb4282a83c2d2936d226ac [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:43:31.782253 20745 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 2b4937775ce34bed9ca35afdca0243b0 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 82186af4c8cb4282a83c2d2936d226ac (127.13.157.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "2b4937775ce34bed9ca35afdca0243b0" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 34721 } } peers { permanent_uuid: "1d39673cf6e84fd9a7012f26f96107e0" member_type: NON_VOTER last_known_addr { host: "127.13.157.189" port: 33117 } attrs { promote: true } } peers { permanent_uuid: "82186af4c8cb4282a83c2d2936d226ac" member_type: NON_VOTER last_known_addr { host: "127.13.157.188" port: 46853 } attrs { promote: true } } }
I20260925 02:43:31.783581 20745 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.13.157.188:46853
I20260925 02:43:31.783589 20880 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 82186af4c8cb4282a83c2d2936d226ac [term 1 FOLLOWER]: Refusing update from remote peer 2b4937775ce34bed9ca35afdca0243b0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260925 02:43:31.783744 20745 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b4937775ce34bed9ca35afdca0243b0 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "2b4937775ce34bed9ca35afdca0243b0" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "2b4937775ce34bed9ca35afdca0243b0" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 34721 } } peers { permanent_uuid: "1d39673cf6e84fd9a7012f26f96107e0" member_type: NON_VOTER last_known_addr { host: "127.13.157.189" port: 33117 } attrs { promote: true } } peers { permanent_uuid: "82186af4c8cb4282a83c2d2936d226ac" member_type: NON_VOTER last_known_addr { host: "127.13.157.188" port: 46853 } attrs { promote: true } } }
I20260925 02:43:31.783828 20745 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b4937775ce34bed9ca35afdca0243b0 [sys.catalog]: This master's current role is: LEADER
I20260925 02:43:31.784122 20744 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2b4937775ce34bed9ca35afdca0243b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "82186af4c8cb4282a83c2d2936d226ac" member_type: NON_VOTER last_known_addr { host: "127.13.157.188" port: 46853 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:43:31.784421 20758 master.cc:537] Master@127.13.157.188:46853 shutting down...
W20260925 02:43:31.785698 20156 consensus_peers.cc:487] T 00000000000000000000000000000000 P 2b4937775ce34bed9ca35afdca0243b0 -> Peer 82186af4c8cb4282a83c2d2936d226ac (127.13.157.188:46853): Couldn't send request to peer 82186af4c8cb4282a83c2d2936d226ac. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260925 02:43:31.788404 20741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:31.781755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.189:33117 (local address 127.0.0.1:56750)
0925 02:43:31.783080 (+ 1325us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:31.783094 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:31.783166 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:31.784241 (+ 1075us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:31.784244 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:31.784264 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:43:31.784350 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:31.784356 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:31.785598 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:31.785601 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:31.786836 (+ 1235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:31.786848 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:31.787987 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:31.787991 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:31.788097 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:31.788143 (+ 46us) client_negotiation.cc:769] Sending connection context
0925 02:43:31.788209 (+ 66us) client_negotiation.cc:240] Negotiation successful
0925 02:43:31.788287 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1247}
I20260925 02:43:31.788724 20910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:31.782338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56750 (local address 127.13.157.189:33117)
0925 02:43:31.783535 (+ 1197us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:31.783550 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:31.783563 (+ 13us) server_negotiation.cc:407] Connection header received
0925 02:43:31.783602 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:31.783604 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:31.783653 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:31.783741 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:43:31.784534 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:31.785470 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:31.787076 (+ 1606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:31.787687 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:31.787797 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:31.787927 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:31.788542 (+ 615us) server_negotiation.cc:299] Negotiation successful
0925 02:43:31.788616 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1127}
I20260925 02:43:31.789463 20909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:31.758843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34820 (local address 127.13.157.188:46853)
0925 02:43:31.758972 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:31.758976 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:31.759153 (+ 177us) server_negotiation.cc:407] Connection header received
0925 02:43:31.759274 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:31.759276 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:31.759309 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:31.759349 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:31.759924 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:31.760593 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:31.761903 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:31.762186 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:31.762245 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:31.783848 (+ 21603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:31.783852 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:31.783854 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:43:31.783876 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:31.787277 (+ 3401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:31.788278 (+ 1001us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:31.788283 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:31.788285 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:31.788384 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:31.788629 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:31.788632 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:31.788633 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:31.788806 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:31.788867 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:31.789175 (+ 308us) server_negotiation.cc:299] Negotiation successful
0925 02:43:31.789277 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":35}
I20260925 02:43:31.789852 20795 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1d39673cf6e84fd9a7012f26f96107e0 [term 1 LEARNER]: Refusing update from remote peer 2b4937775ce34bed9ca35afdca0243b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260925 02:43:31.790268 20932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:31.758580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.188:46853 (local address 127.0.0.1:34820)
0925 02:43:31.759090 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:31.759101 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:31.759197 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:31.759456 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:31.759459 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:31.759705 (+ 246us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:31.759816 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:31.759823 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:31.760688 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:31.760690 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:31.761806 (+ 1116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:31.761814 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:31.762355 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:31.762358 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:31.762435 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:31.782499 (+ 20064us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:31.782514 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:31.783734 (+ 1220us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:43:31.787617 (+ 3883us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:31.787620 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:31.787622 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:31.787842 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:31.788473 (+ 631us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:31.788475 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:31.788477 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:31.788550 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:31.788920 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:43:31.788923 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:43:31.788998 (+ 75us) client_negotiation.cc:769] Sending connection context
0925 02:43:31.789611 (+ 613us) client_negotiation.cc:240] Negotiation successful
0925 02:43:31.790110 (+ 499us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":450,"mutex_wait_us":38,"thread_start_us":79,"threads_started":1}
I20260925 02:43:31.790234 20744 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2b4937775ce34bed9ca35afdca0243b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d39673cf6e84fd9a7012f26f96107e0" member_type: NON_VOTER last_known_addr { host: "127.13.157.189" port: 33117 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260925 02:43:31.790319 20758 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 82186af4c8cb4282a83c2d2936d226ac [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 02:43:31.790599 20758 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 82186af4c8cb4282a83c2d2936d226ac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:43:31.790697 20758 tablet_replica.cc:330] T 00000000000000000000000000000000 P 82186af4c8cb4282a83c2d2936d226ac: stopping tablet replica
I20260925 02:43:31.792207 20795 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1d39673cf6e84fd9a7012f26f96107e0 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 82186af4c8cb4282a83c2d2936d226ac (127.13.157.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "2b4937775ce34bed9ca35afdca0243b0" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 34721 } } peers { permanent_uuid: "1d39673cf6e84fd9a7012f26f96107e0" member_type: NON_VOTER last_known_addr { host: "127.13.157.189" port: 33117 } attrs { promote: true } } peers { permanent_uuid: "82186af4c8cb4282a83c2d2936d226ac" member_type: NON_VOTER last_known_addr { host: "127.13.157.188" port: 46853 } attrs { promote: true } } }
I20260925 02:43:31.794063 20936 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d39673cf6e84fd9a7012f26f96107e0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2b4937775ce34bed9ca35afdca0243b0. Latest consensus state: current_term: 1 leader_uuid: "2b4937775ce34bed9ca35afdca0243b0" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "2b4937775ce34bed9ca35afdca0243b0" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 34721 } } peers { permanent_uuid: "1d39673cf6e84fd9a7012f26f96107e0" member_type: NON_VOTER last_known_addr { host: "127.13.157.189" port: 33117 } attrs { promote: true } } peers { permanent_uuid: "82186af4c8cb4282a83c2d2936d226ac" member_type: NON_VOTER last_known_addr { host: "127.13.157.188" port: 46853 } attrs { promote: true } } }
I20260925 02:43:31.794221 20936 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d39673cf6e84fd9a7012f26f96107e0 [sys.catalog]: This master's current role is: LEARNER
I20260925 02:43:31.794179 20827 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d39673cf6e84fd9a7012f26f96107e0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2b4937775ce34bed9ca35afdca0243b0" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "2b4937775ce34bed9ca35afdca0243b0" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 34721 } } peers { permanent_uuid: "1d39673cf6e84fd9a7012f26f96107e0" member_type: NON_VOTER last_known_addr { host: "127.13.157.189" port: 33117 } attrs { promote: true } } peers { permanent_uuid: "82186af4c8cb4282a83c2d2936d226ac" member_type: NON_VOTER last_known_addr { host: "127.13.157.188" port: 46853 } attrs { promote: true } } }
I20260925 02:43:31.794289 20827 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d39673cf6e84fd9a7012f26f96107e0 [sys.catalog]: This master's current role is: LEARNER
I20260925 02:43:31.796468 20744 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 2b4937775ce34bed9ca35afdca0243b0: attempting to promote NON_VOTER 1d39673cf6e84fd9a7012f26f96107e0 to VOTER
I20260925 02:43:31.796902 20744 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2b4937775ce34bed9ca35afdca0243b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "2b4937775ce34bed9ca35afdca0243b0" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 34721 } } peers { permanent_uuid: "1d39673cf6e84fd9a7012f26f96107e0" member_type: VOTER last_known_addr { host: "127.13.157.189" port: 33117 } attrs { promote: false } } peers { permanent_uuid: "82186af4c8cb4282a83c2d2936d226ac" member_type: NON_VOTER last_known_addr { host: "127.13.157.188" port: 46853 } attrs { promote: true } }
I20260925 02:43:31.798072 20741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:31.797766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.188:46853 (local address 127.0.0.1:34824)
0925 02:43:31.797923 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:31.797995 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.188:46853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260925 02:43:31.798839 20741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:31.798598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.188:46853 (local address 127.0.0.1:34836)
0925 02:43:31.798732 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:31.798777 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.188:46853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
W20260925 02:43:31.799084 20156 consensus_peers.cc:487] T 00000000000000000000000000000000 P 2b4937775ce34bed9ca35afdca0243b0 -> Peer 82186af4c8cb4282a83c2d2936d226ac (127.13.157.188:46853): Couldn't send request to peer 82186af4c8cb4282a83c2d2936d226ac. Status: Network error: Client connection negotiation failed: client connection to 127.13.157.188:46853: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260925 02:43:31.799117 20932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:31.798653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.188:46853 (local address 127.0.0.1:34844)
0925 02:43:31.798857 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:31.799051 (+ 194us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.188:46853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260925 02:43:31.799350 20795 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1d39673cf6e84fd9a7012f26f96107e0 [term 1 LEARNER]: Refusing update from remote peer 2b4937775ce34bed9ca35afdca0243b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260925 02:43:31.799702 20743 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2b4937775ce34bed9ca35afdca0243b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d39673cf6e84fd9a7012f26f96107e0" member_type: VOTER last_known_addr { host: "127.13.157.189" port: 33117 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260925 02:43:31.800783 20758 master.cc:552] Master@127.13.157.188:46853 shutdown complete.
I20260925 02:43:31.800834 20758 master_runner.cc:305] Clearing existing system tablet
I20260925 02:43:31.801218 20758 fs_manager.cc:359] Metadata directory not provided
I20260925 02:43:31.801275 20758 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:43:31.801367 20743 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 2b4937775ce34bed9ca35afdca0243b0 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 1d39673cf6e84fd9a7012f26f96107e0 (127.13.157.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "2b4937775ce34bed9ca35afdca0243b0" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 34721 } } peers { permanent_uuid: "1d39673cf6e84fd9a7012f26f96107e0" member_type: VOTER last_known_addr { host: "127.13.157.189" port: 33117 } attrs { promote: false } } peers { permanent_uuid: "82186af4c8cb4282a83c2d2936d226ac" member_type: NON_VOTER last_known_addr { host: "127.13.157.188" port: 46853 } attrs { promote: true } } }
I20260925 02:43:31.802565 20758 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 02:43:31.802747 20743 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b4937775ce34bed9ca35afdca0243b0 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "2b4937775ce34bed9ca35afdca0243b0" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "2b4937775ce34bed9ca35afdca0243b0" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 34721 } } peers { permanent_uuid: "1d39673cf6e84fd9a7012f26f96107e0" member_type: VOTER last_known_addr { host: "127.13.157.189" port: 33117 } attrs { promote: false } } peers { permanent_uuid: "82186af4c8cb4282a83c2d2936d226ac" member_type: NON_VOTER last_known_addr { host: "127.13.157.188" port: 46853 } attrs { promote: true } } }
I20260925 02:43:31.802840 20743 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b4937775ce34bed9ca35afdca0243b0 [sys.catalog]: This master's current role is: LEADER
I20260925 02:43:31.803344 20938 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:31.803506 20758 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:43:31.803544 20743 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b4937775ce34bed9ca35afdca0243b0 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2b4937775ce34bed9ca35afdca0243b0" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "2b4937775ce34bed9ca35afdca0243b0" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 34721 } } peers { permanent_uuid: "1d39673cf6e84fd9a7012f26f96107e0" member_type: VOTER last_known_addr { host: "127.13.157.189" port: 33117 } attrs { promote: false } } peers { permanent_uuid: "82186af4c8cb4282a83c2d2936d226ac" member_type: NON_VOTER last_known_addr { host: "127.13.157.188" port: 46853 } attrs { promote: true } } }
I20260925 02:43:31.803619 20758 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260925 02:43:31.803637 20743 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b4937775ce34bed9ca35afdca0243b0 [sys.catalog]: This master's current role is: LEADER
I20260925 02:43:31.803649 20758 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/master-2/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/master-2/wal
uuid: "82186af4c8cb4282a83c2d2936d226ac"
format_stamp: "Formatted at 2026-09-25 02:43:31 on dist-test-slave-kznw"
server_key: "b00244ae50c3e58ac66c51f1ca23940d"
server_key_iv: "077ff01cbe6f08d79a9c15d62d0029e1"
server_key_version: "encryptionkey@0"
I20260925 02:43:31.804111 20795 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1d39673cf6e84fd9a7012f26f96107e0 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 1d39673cf6e84fd9a7012f26f96107e0 (127.13.157.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "2b4937775ce34bed9ca35afdca0243b0" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 34721 } } peers { permanent_uuid: "1d39673cf6e84fd9a7012f26f96107e0" member_type: VOTER last_known_addr { host: "127.13.157.189" port: 33117 } attrs { promote: false } } peers { permanent_uuid: "82186af4c8cb4282a83c2d2936d226ac" member_type: NON_VOTER last_known_addr { host: "127.13.157.188" port: 46853 } attrs { promote: true } } }
I20260925 02:43:31.805662 20827 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d39673cf6e84fd9a7012f26f96107e0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2b4937775ce34bed9ca35afdca0243b0" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "2b4937775ce34bed9ca35afdca0243b0" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 34721 } } peers { permanent_uuid: "1d39673cf6e84fd9a7012f26f96107e0" member_type: VOTER last_known_addr { host: "127.13.157.189" port: 33117 } attrs { promote: false } } peers { permanent_uuid: "82186af4c8cb4282a83c2d2936d226ac" member_type: NON_VOTER last_known_addr { host: "127.13.157.188" port: 46853 } attrs { promote: true } } }
I20260925 02:43:31.805764 20827 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d39673cf6e84fd9a7012f26f96107e0 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 02:43:31.806187 20758 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 82186af4c8cb4282a83c2d2936d226ac: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:43:31.808409 20758 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 82186af4c8cb4282a83c2d2936d226ac: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260925 02:43:31.808497 20758 log.cc:1196] T 00000000000000000000000000000000 P 82186af4c8cb4282a83c2d2936d226ac: Deleting WAL directory at /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260925 02:43:31.808729 20758 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 82186af4c8cb4282a83c2d2936d226ac: Deleting consensus metadata
I20260925 02:43:31.808878 20758 master_runner.cc:315] Copying system tablet from 127.13.157.190:34721
I20260925 02:43:31.810038 20758 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 82186af4c8cb4282a83c2d2936d226ac: tablet copy: Beginning tablet copy session from remote peer at address 127.13.157.190:34721
I20260925 02:43:31.818552 20211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:31.810569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32834 (local address 127.13.157.190:34721)
0925 02:43:31.810675 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:31.810679 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:31.810830 (+ 151us) server_negotiation.cc:407] Connection header received
0925 02:43:31.810993 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:31.810996 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:31.811045 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:31.811127 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:31.811586 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:31.812380 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:31.813546 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:31.813883 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:31.813976 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:31.815257 (+ 1281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:31.815261 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:31.815263 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:43:31.815290 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:31.816916 (+ 1626us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:31.817366 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:31.817370 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:31.817372 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:31.817457 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:31.817728 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:31.817731 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:31.817733 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:31.817935 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:31.818028 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:31.818282 (+ 254us) server_negotiation.cc:299] Negotiation successful
0925 02:43:31.818392 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260925 02:43:31.818558 20944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:31.810500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:34721 (local address 127.0.0.1:32834)
0925 02:43:31.810746 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:31.810759 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:31.810859 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:31.811151 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:31.811154 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:31.811355 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:31.811479 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:31.811486 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:31.812483 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:31.812486 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:31.813428 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:31.813439 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:31.813987 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:31.813991 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:31.814094 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:31.814625 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:31.814639 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:31.815092 (+ 453us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:43:31.817018 (+ 1926us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:31.817028 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:31.817030 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:31.817272 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:31.817562 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:31.817565 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:31.817567 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:31.817650 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:31.818026 (+ 376us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:43:31.818030 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:43:31.818126 (+ 96us) client_negotiation.cc:769] Sending connection context
0925 02:43:31.818224 (+ 98us) client_negotiation.cc:240] Negotiation successful
0925 02:43:31.818355 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":60,"threads_started":1}
I20260925 02:43:31.819370 20192 tablet_copy_service.cc:140] P 2b4937775ce34bed9ca35afdca0243b0: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 82186af4c8cb4282a83c2d2936d226ac ({username='oryx', principal='oryx/127.13.157.188@KRBTEST.COM'} at 127.0.0.1:32834)
I20260925 02:43:31.819465 20192 tablet_copy_service.cc:161] P 2b4937775ce34bed9ca35afdca0243b0: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 82186af4c8cb4282a83c2d2936d226ac at {username='oryx', principal='oryx/127.13.157.188@KRBTEST.COM'} at 127.0.0.1:32834: session id = 82186af4c8cb4282a83c2d2936d226ac-00000000000000000000000000000000
I20260925 02:43:31.820509 20192 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 2b4937775ce34bed9ca35afdca0243b0: Tablet Copy: opened 0 blocks and 1 log segments
I20260925 02:43:31.821754 20758 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260925 02:43:31.824131 20758 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 82186af4c8cb4282a83c2d2936d226ac: tablet copy: Starting download of 0 data blocks...
I20260925 02:43:31.824306 20758 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 82186af4c8cb4282a83c2d2936d226ac: tablet copy: Starting download of 1 WAL segments...
I20260925 02:43:31.825666 20758 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 82186af4c8cb4282a83c2d2936d226ac: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260925 02:43:31.827416 20192 tablet_copy_service.cc:342] P 2b4937775ce34bed9ca35afdca0243b0: Request end of tablet copy session 82186af4c8cb4282a83c2d2936d226ac-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.13.157.188@KRBTEST.COM'} at 127.0.0.1:32834
I20260925 02:43:31.827503 20192 tablet_copy_service.cc:433] P 2b4937775ce34bed9ca35afdca0243b0: ending tablet copy session 82186af4c8cb4282a83c2d2936d226ac-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 82186af4c8cb4282a83c2d2936d226ac
I20260925 02:43:31.834611 20758 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:43:31.836494 20948 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:43:31.836815 20758 server_base.cc:1034] running on GCE node
W20260925 02:43:31.838404 20945 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:31.839737 20946 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:43:31.840034 20758 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:43:31.840350 20758 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:43:31.841492 20758 hybrid_clock.cc:648] HybridClock initialized: now 1790304211841469 us; error 39 us; skew 500 ppm
Sep 25 02:43:31 dist-test-slave-kznw krb5kdc[20127](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304211, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.157.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:43:31.843109 20758 init.cc:377] Logged in from keytab as oryx/127.13.157.188@KRBTEST.COM (short username oryx)
I20260925 02:43:31.843608 20758 webserver.cc:460] Webserver started at http://127.13.157.188:44389/ using document root <none> and password file <none>
I20260925 02:43:31.843786 20758 fs_manager.cc:359] Metadata directory not provided
I20260925 02:43:31.843825 20758 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:43:31.845243 20758 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 02:43:31.846275 20956 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:31.846575 20758 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 02:43:31.846688 20758 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/master-2/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/master-2/wal
uuid: "82186af4c8cb4282a83c2d2936d226ac"
format_stamp: "Formatted at 2026-09-25 02:43:31 on dist-test-slave-kznw"
server_key: "b00244ae50c3e58ac66c51f1ca23940d"
server_key_iv: "077ff01cbe6f08d79a9c15d62d0029e1"
server_key_version: "encryptionkey@0"
I20260925 02:43:31.846783 20758 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:43:31.871981 20758 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:43:31.874226 20758 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:43:31.878978 20758 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.188:46853
I20260925 02:43:31.879102 21009 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.188:46853 every 8 connection(s)
I20260925 02:43:31.879581 20758 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/master-2/data/info.pb
I20260925 02:43:31.881510 21010 sys_catalog.cc:263] Verifying existing consensus state
I20260925 02:43:31.882668 21010 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82186af4c8cb4282a83c2d2936d226ac: Bootstrap starting.
I20260925 02:43:31.894335 21010 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82186af4c8cb4282a83c2d2936d226ac: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 02:43:31.894682 21010 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82186af4c8cb4282a83c2d2936d226ac: Bootstrap complete.
I20260925 02:43:31.895301 21010 raft_consensus.cc:348] T 00000000000000000000000000000000 P 82186af4c8cb4282a83c2d2936d226ac [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "2b4937775ce34bed9ca35afdca0243b0" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 34721 } } peers { permanent_uuid: "1d39673cf6e84fd9a7012f26f96107e0" member_type: VOTER last_known_addr { host: "127.13.157.189" port: 33117 } attrs { promote: false } } peers { permanent_uuid: "82186af4c8cb4282a83c2d2936d226ac" member_type: NON_VOTER last_known_addr { host: "127.13.157.188" port: 46853 } attrs { promote: true } }
I20260925 02:43:31.895442 21010 raft_consensus.cc:729] T 00000000000000000000000000000000 P 82186af4c8cb4282a83c2d2936d226ac [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 82186af4c8cb4282a83c2d2936d226ac, State: Initialized, Role: LEARNER
I20260925 02:43:31.895536 21010 consensus_queue.cc:260] T 00000000000000000000000000000000 P 82186af4c8cb4282a83c2d2936d226ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "2b4937775ce34bed9ca35afdca0243b0" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 34721 } } peers { permanent_uuid: "1d39673cf6e84fd9a7012f26f96107e0" member_type: VOTER last_known_addr { host: "127.13.157.189" port: 33117 } attrs { promote: false } } peers { permanent_uuid: "82186af4c8cb4282a83c2d2936d226ac" member_type: NON_VOTER last_known_addr { host: "127.13.157.188" port: 46853 } attrs { promote: true } }
I20260925 02:43:31.895993 21010 sys_catalog.cc:564] T 00000000000000000000000000000000 P 82186af4c8cb4282a83c2d2936d226ac [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:43:31.895948 21013 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82186af4c8cb4282a83c2d2936d226ac [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "2b4937775ce34bed9ca35afdca0243b0" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 34721 } } peers { permanent_uuid: "1d39673cf6e84fd9a7012f26f96107e0" member_type: VOTER last_known_addr { host: "127.13.157.189" port: 33117 } attrs { promote: false } } peers { permanent_uuid: "82186af4c8cb4282a83c2d2936d226ac" member_type: NON_VOTER last_known_addr { host: "127.13.157.188" port: 46853 } attrs { promote: true } } }
I20260925 02:43:31.896196 21013 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82186af4c8cb4282a83c2d2936d226ac [sys.catalog]: This master's current role is: LEARNER
I20260925 02:43:31.900557 21024 catalog_manager.cc:1259] Loaded cluster ID: 19dd28aa2b484a96bcdd464cb2d07319
I20260925 02:43:31.900622 21024 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 82186af4c8cb4282a83c2d2936d226ac: loading cluster ID for follower catalog manager: success
I20260925 02:43:31.902729 21024 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 82186af4c8cb4282a83c2d2936d226ac: acquiring CA information for follower catalog manager: success
I20260925 02:43:31.904148 21024 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 82186af4c8cb4282a83c2d2936d226ac: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260925 02:43:31.928721 20934 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:43:31.928979 20934 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:43:31.929023 20934 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:43:31.932996 20934 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:43:31.933070 20934 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:43:31.933131 20934 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.157.129
I20260925 02:43:31.938560 20934 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:46123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/krb5kdc/oryx_127.13.157.129.keytab
--principal=oryx/127.13.157.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.157.129:46073
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.157.129
--webserver_port=39687
--tserver_master_addrs=127.13.157.190:34721,127.13.157.189:33117,127.13.157.188:46853
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20934
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.157.129
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:43:31.939725 20934 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:43:31.940737 20934 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 02:43:31.950402 20934 server_base.cc:1034] running on GCE node
W20260925 02:43:31.950363 21030 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:31.950354 21029 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:31.950351 21032 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:43:31.951087 20934 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:43:31.951733 20934 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:43:31.952878 20934 hybrid_clock.cc:648] HybridClock initialized: now 1790304211952826 us; error 62 us; skew 500 ppm
Sep 25 02:43:31 dist-test-slave-kznw krb5kdc[20127](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304211, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.157.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:43:31.955803 20934 init.cc:377] Logged in from keytab as oryx/127.13.157.129@KRBTEST.COM (short username oryx)
I20260925 02:43:31.956714 20934 webserver.cc:460] Webserver started at http://127.13.157.129:39687/ using document root <none> and password file <none>
I20260925 02:43:31.957233 20934 fs_manager.cc:359] Metadata directory not provided
I20260925 02:43:31.957312 20934 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:43:31.961366 20934 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260925 02:43:31.963371 21039 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:31.964579 20934 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260925 02:43:31.964684 20934 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/ts-0/wal
uuid: "15b827f1b5534fafa26e3142b9e9d192"
format_stamp: "Formatted at 2026-09-25 02:43:30 on dist-test-slave-kznw"
server_key: "54e5127bd46f635a98e0f1a79eb3340f"
server_key_iv: "19ff5e85c1d80525f659aa5ec72632b8"
server_key_version: "encryptionkey@0"
I20260925 02:43:31.965220 20934 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:43:31.980087 20502 heartbeater.cc:503] Master 127.13.157.190:34721 was elected leader, sending a full tablet report...
I20260925 02:43:31.991564 20934 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:43:31.995065 20934 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:43:31.995234 20934 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:43:31.995824 20934 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:43:31.996770 20934 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:43:31.996824 20934 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:31.996881 20934 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:43:31.996908 20934 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:32.005959 20934 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.129:46073
I20260925 02:43:32.006089 21155 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.129:46073 every 8 connection(s)
I20260925 02:43:32.007022 20934 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/ts-0/data/info.pb
I20260925 02:43:32.008172 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 20934
I20260925 02:43:32.008345 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 20371
I20260925 02:43:32.022802 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.157.130:36015
--local_ip_for_outbound_sockets=127.13.157.130
--tserver_master_addrs=127.13.157.190:34721
--webserver_port=37403
--webserver_interface=127.13.157.130
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/krb5kdc/oryx_127.13.157.130.keytab
--principal=oryx/127.13.157.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:46123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.13.157.190:34721,127.13.157.189:33117
--tserver_master_addrs=127.13.157.190:34721,127.13.157.189:33117,127.13.157.188:46853 with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 25 02:43:32 dist-test-slave-kznw krb5kdc[20127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304211, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.157.129@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM
Sep 25 02:43:32 dist-test-slave-kznw krb5kdc[20127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304211, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.157.129@KRBTEST.COM for oryx/127.13.157.188@KRBTEST.COM
Sep 25 02:43:32 dist-test-slave-kznw krb5kdc[20127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304211, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.157.129@KRBTEST.COM for oryx/127.13.157.189@KRBTEST.COM
I20260925 02:43:32.031940 20211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:32.019304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.129:46155 (local address 127.13.157.190:34721)
0925 02:43:32.019413 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:32.019417 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:32.019653 (+ 236us) server_negotiation.cc:407] Connection header received
0925 02:43:32.020314 (+ 661us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:32.020317 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:32.020376 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:32.020455 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:32.022287 (+ 1832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:32.023088 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:32.024699 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:32.024969 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:32.025067 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:32.027964 (+ 2897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:32.027968 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:32.027970 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:43:32.028001 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:32.029467 (+ 1466us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:32.030090 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:32.030095 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:32.030096 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:32.030188 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:32.030622 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:32.030625 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:32.030627 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:32.030854 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:32.030954 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:32.031650 (+ 696us) server_negotiation.cc:299] Negotiation successful
0925 02:43:32.031770 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260925 02:43:32.032474 21161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:32.018137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.129:39739 (local address 127.13.157.188:46853)
0925 02:43:32.018457 (+ 320us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:32.018461 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:32.019444 (+ 983us) server_negotiation.cc:407] Connection header received
0925 02:43:32.020604 (+ 1160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:32.020608 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:32.020658 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:32.020740 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:32.022293 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:32.023323 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:32.024566 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:32.024867 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:32.024962 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:32.028397 (+ 3435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:32.028401 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:32.028404 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:43:32.028434 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:32.030159 (+ 1725us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:32.030824 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:32.030828 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:32.030830 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:32.030944 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:32.031263 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:32.031266 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:32.031268 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:32.031482 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:32.031558 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:32.032176 (+ 618us) server_negotiation.cc:299] Negotiation successful
0925 02:43:32.032302 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":231,"thread_start_us":117,"threads_started":1}
I20260925 02:43:32.033440 21160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:32.010523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.188:46853 (local address 127.13.157.129:39739)
0925 02:43:32.019014 (+ 8491us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:32.019089 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:32.020092 (+ 1003us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:32.020864 (+ 772us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:32.020867 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:32.021308 (+ 441us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:32.022133 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:32.022151 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:32.023441 (+ 1290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:32.023443 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:32.024439 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:32.024449 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:32.025129 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:32.025132 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:32.025307 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:32.026658 (+ 1351us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:32.026670 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:32.028240 (+ 1570us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:43:32.030273 (+ 2033us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:32.030276 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:32.030278 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:32.030706 (+ 428us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:32.031071 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:32.031074 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:32.031076 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:32.031162 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:32.031633 (+ 471us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:43:32.031636 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:43:32.031738 (+ 102us) client_negotiation.cc:769] Sending connection context
0925 02:43:32.031884 (+ 146us) client_negotiation.cc:240] Negotiation successful
0925 02:43:32.032798 (+ 914us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8188,"mutex_wait_us":110,"spinlock_wait_cycles":102656,"thread_start_us":672,"threads_started":1}
I20260925 02:43:32.033723 21163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:32.018898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:34721 (local address 127.13.157.129:46155)
0925 02:43:32.019576 (+ 678us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:32.019588 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:32.020097 (+ 509us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:32.020668 (+ 571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:32.020678 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:32.021310 (+ 632us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:32.022135 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:32.022152 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:32.023299 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:32.023302 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:32.024574 (+ 1272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:32.024582 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:32.025226 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:32.025229 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:32.025314 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:32.025970 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:32.025998 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:32.027750 (+ 1752us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:43:32.029608 (+ 1858us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:32.029613 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:32.029630 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:32.029974 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:32.030308 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:32.030311 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:32.030313 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:32.030506 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:32.031039 (+ 533us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:43:32.031045 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:43:32.031359 (+ 314us) client_negotiation.cc:769] Sending connection context
0925 02:43:32.032100 (+ 741us) client_negotiation.cc:240] Negotiation successful
0925 02:43:32.032350 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":595,"mutex_wait_us":70,"spinlock_wait_cycles":70528,"thread_start_us":91,"threads_started":1}
I20260925 02:43:32.034837 21162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:32.019088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.189:33117 (local address 127.13.157.129:49967)
0925 02:43:32.024447 (+ 5359us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:32.024461 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:32.024616 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:32.024887 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:32.024890 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:32.025103 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:32.025223 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:32.025229 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:32.026144 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:32.026148 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:32.026931 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:32.026943 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:32.027413 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:32.027416 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:32.027501 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:32.027929 (+ 428us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:32.027942 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:32.029287 (+ 1345us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:43:32.031592 (+ 2305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:32.031595 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:32.031597 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:32.031829 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:32.033270 (+ 1441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:32.033272 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:32.033274 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:32.033361 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:32.034405 (+ 1044us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:43:32.034408 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:43:32.034480 (+ 72us) client_negotiation.cc:769] Sending connection context
0925 02:43:32.034561 (+ 81us) client_negotiation.cc:240] Negotiation successful
0925 02:43:32.034686 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5290,"thread_start_us":137,"threads_started":1}
I20260925 02:43:32.034930 20910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:32.020231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.129:49967 (local address 127.13.157.189:33117)
0925 02:43:32.020356 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:32.020360 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:32.024545 (+ 4185us) server_negotiation.cc:407] Connection header received
0925 02:43:32.024722 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:32.024726 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:32.024785 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:32.024862 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:32.025344 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:32.026024 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:32.027040 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:32.027310 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:32.027409 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:32.029619 (+ 2210us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:32.029623 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:32.029625 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:43:32.029651 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:32.031308 (+ 1657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:32.031962 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:32.031966 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:32.031968 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:32.033151 (+ 1183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:32.033460 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:32.033463 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:32.033465 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:32.034263 (+ 798us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:32.034335 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:32.034632 (+ 297us) server_negotiation.cc:299] Negotiation successful
0925 02:43:32.034745 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260925 02:43:32.035181 21157 heartbeater.cc:347] Connected to a master server at 127.13.157.190:34721
I20260925 02:43:32.035182 21156 heartbeater.cc:347] Connected to a master server at 127.13.157.188:46853
I20260925 02:43:32.035440 21157 heartbeater.cc:464] Registering TS with master...
I20260925 02:43:32.035439 21156 heartbeater.cc:464] Registering TS with master...
I20260925 02:43:32.035830 21158 heartbeater.cc:347] Connected to a master server at 127.13.157.189:33117
I20260925 02:43:32.035936 21158 heartbeater.cc:464] Registering TS with master...
I20260925 02:43:32.036113 21157 heartbeater.cc:511] Master 127.13.157.190:34721 requested a full tablet report, sending...
I20260925 02:43:32.036113 21156 heartbeater.cc:511] Master 127.13.157.188:46853 requested a full tablet report, sending...
I20260925 02:43:32.036135 21158 heartbeater.cc:511] Master 127.13.157.189:33117 requested a full tablet report, sending...
I20260925 02:43:32.037453 20171 ts_manager.cc:194] Re-registered known tserver with Master: 15b827f1b5534fafa26e3142b9e9d192 (127.13.157.129:46073)
I20260925 02:43:32.037678 20973 ts_manager.cc:194] Registered new tserver with Master: 15b827f1b5534fafa26e3142b9e9d192 (127.13.157.129:46073)
I20260925 02:43:32.037676 20787 ts_manager.cc:194] Registered new tserver with Master: 15b827f1b5534fafa26e3142b9e9d192 (127.13.157.129:46073)
I20260925 02:43:32.038214 20171 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.157.129@KRBTEST.COM'} at 127.13.157.129:46155
W20260925 02:43:32.138182 21164 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:43:32.138434 21164 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:43:32.138484 21164 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:43:32.142419 21164 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:43:32.142486 21164 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:43:32.142547 21164 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.157.130
I20260925 02:43:32.147884 21164 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:46123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/krb5kdc/oryx_127.13.157.130.keytab
--principal=oryx/127.13.157.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.157.130:36015
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.157.130
--webserver_port=37403
--tserver_master_addrs=127.13.157.190:34721,127.13.157.189:33117,127.13.157.188:46853
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21164
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.157.130
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:43:32.148872 21164 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:43:32.149823 21164 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:43:32.159385 21169 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:32.159421 21172 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:32.159385 21170 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:43:32.159813 21164 server_base.cc:1034] running on GCE node
I20260925 02:43:32.160207 21164 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:43:32.160837 21164 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:43:32.162002 21164 hybrid_clock.cc:648] HybridClock initialized: now 1790304212161963 us; error 50 us; skew 500 ppm
Sep 25 02:43:32 dist-test-slave-kznw krb5kdc[20127](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304212, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.157.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:43:32.164768 21164 init.cc:377] Logged in from keytab as oryx/127.13.157.130@KRBTEST.COM (short username oryx)
I20260925 02:43:32.165624 21164 webserver.cc:460] Webserver started at http://127.13.157.130:37403/ using document root <none> and password file <none>
I20260925 02:43:32.166150 21164 fs_manager.cc:359] Metadata directory not provided
I20260925 02:43:32.166232 21164 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:43:32.170408 21164 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260925 02:43:32.172606 21179 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:32.173938 21164 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260925 02:43:32.174117 21164 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/ts-1/wal
uuid: "03b8c1c27f5b471498f53d8d326b61db"
format_stamp: "Formatted at 2026-09-25 02:43:30 on dist-test-slave-kznw"
server_key: "1ae251b5050bb94b14cce9f651bc000a"
server_key_iv: "d3b3727d1341d3b77446586d4e259bbd"
server_key_version: "encryptionkey@0"
I20260925 02:43:32.174651 21164 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:43:32.189299 20640 heartbeater.cc:503] Master 127.13.157.190:34721 was elected leader, sending a full tablet report...
I20260925 02:43:32.191011 21164 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:43:32.193993 21164 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:43:32.194157 21164 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:43:32.194684 21164 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:43:32.195503 21164 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:43:32.195587 21164 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:32.195637 21164 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:43:32.195657 21164 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:32.203905 21164 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.130:36015
I20260925 02:43:32.203987 21294 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.130:36015 every 8 connection(s)
I20260925 02:43:32.204898 21164 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/ts-1/data/info.pb
I20260925 02:43:32.213636 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 21164
I20260925 02:43:32.213850 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 20507
Sep 25 02:43:32 dist-test-slave-kznw krb5kdc[20127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304212, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.157.130@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM
Sep 25 02:43:32 dist-test-slave-kznw krb5kdc[20127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304212, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.157.130@KRBTEST.COM for oryx/127.13.157.189@KRBTEST.COM
Sep 25 02:43:32 dist-test-slave-kznw krb5kdc[20127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304212, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.157.130@KRBTEST.COM for oryx/127.13.157.188@KRBTEST.COM
I20260925 02:43:32.221825 20910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:32.206711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.130:58395 (local address 127.13.157.189:33117)
0925 02:43:32.206846 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:32.206850 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:32.207878 (+ 1028us) server_negotiation.cc:407] Connection header received
0925 02:43:32.209003 (+ 1125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:32.209006 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:32.209105 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:32.209184 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:32.210663 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:32.211248 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:32.213206 (+ 1958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:32.213546 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:32.213707 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:32.217171 (+ 3464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:32.217176 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:32.217178 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:43:32.217206 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:32.219210 (+ 2004us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:32.219879 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:32.219883 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:32.219884 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:32.219967 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:32.220418 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:32.220421 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:32.220423 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:32.220673 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:32.220745 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:32.221517 (+ 772us) server_negotiation.cc:299] Negotiation successful
0925 02:43:32.221654 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260925 02:43:32.221841 21161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:32.208242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.130:33619 (local address 127.13.157.188:46853)
0925 02:43:32.208381 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:32.208386 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:32.208403 (+ 17us) server_negotiation.cc:407] Connection header received
0925 02:43:32.209308 (+ 905us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:32.209311 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:32.209361 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:32.209438 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:32.210622 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:32.211354 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:32.213153 (+ 1799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:32.213527 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:32.213628 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:32.217977 (+ 4349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:32.217982 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:32.217984 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:43:32.218015 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:32.219594 (+ 1579us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:32.220145 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:32.220149 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:32.220151 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:32.220248 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:32.220536 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:32.220539 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:32.220541 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:32.220740 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:32.220803 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:32.221504 (+ 701us) server_negotiation.cc:299] Negotiation successful
0925 02:43:32.221654 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260925 02:43:32.222105 20211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:32.206758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.130:37209 (local address 127.13.157.190:34721)
0925 02:43:32.207060 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:32.207065 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:32.208133 (+ 1068us) server_negotiation.cc:407] Connection header received
0925 02:43:32.208900 (+ 767us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:32.208904 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:32.208985 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:32.209066 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:32.211058 (+ 1992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:32.211705 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:32.212900 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:32.213130 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:32.213230 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:32.216904 (+ 3674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:32.216908 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:32.216911 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:43:32.216939 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:32.219334 (+ 2395us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:32.219900 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:32.219904 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:32.219906 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:32.219994 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:32.220367 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:32.220370 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:32.220372 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:32.220688 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:32.220805 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:32.221828 (+ 1023us) server_negotiation.cc:299] Negotiation successful
0925 02:43:32.221946 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":203}
I20260925 02:43:32.223541 21299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:32.207200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.189:33117 (local address 127.13.157.130:58395)
0925 02:43:32.207693 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:32.207746 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:32.208686 (+ 940us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:32.209258 (+ 572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:32.209265 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:32.209864 (+ 599us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:32.210464 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:32.210479 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:32.211388 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:32.211391 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:32.213023 (+ 1632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:32.213035 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:32.214362 (+ 1327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:32.214366 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:32.214455 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:32.214892 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:32.214930 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:32.217001 (+ 2071us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:43:32.219338 (+ 2337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:32.219344 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:32.219362 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:32.219770 (+ 408us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:32.220229 (+ 459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:32.220231 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:32.220233 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:32.220318 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:32.220828 (+ 510us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:43:32.220835 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:43:32.221210 (+ 375us) client_negotiation.cc:769] Sending connection context
0925 02:43:32.222704 (+ 1494us) client_negotiation.cc:240] Negotiation successful
0925 02:43:32.222858 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"mutex_wait_us":516,"spinlock_wait_cycles":21504,"thread_start_us":123,"threads_started":1}
I20260925 02:43:32.223817 21300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:32.207200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:34721 (local address 127.13.157.130:37209)
0925 02:43:32.208048 (+ 848us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:32.208061 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:32.208692 (+ 631us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:32.209259 (+ 567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:32.209267 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:32.209895 (+ 628us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:32.210466 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:32.210480 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:32.211785 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:32.211788 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:32.212764 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:32.212774 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:32.213263 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:32.213267 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:32.213453 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:32.214339 (+ 886us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:32.214366 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:32.216716 (+ 2350us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:43:32.219454 (+ 2738us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:32.219457 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:32.219459 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:32.219787 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:32.220087 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:32.220089 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:32.220091 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:32.220270 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:32.220853 (+ 583us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:43:32.220856 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:43:32.221210 (+ 354us) client_negotiation.cc:769] Sending connection context
0925 02:43:32.221415 (+ 205us) client_negotiation.cc:240] Negotiation successful
0925 02:43:32.222299 (+ 884us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":756,"mutex_wait_us":183,"spinlock_wait_cycles":65280,"thread_start_us":136,"threads_started":1}
I20260925 02:43:32.223981 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.157.131:33075
--local_ip_for_outbound_sockets=127.13.157.131
--tserver_master_addrs=127.13.157.190:34721
--webserver_port=37925
--webserver_interface=127.13.157.131
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/krb5kdc/oryx_127.13.157.131.keytab
--principal=oryx/127.13.157.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:46123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.13.157.190:34721,127.13.157.189:33117
--tserver_master_addrs=127.13.157.190:34721,127.13.157.189:33117,127.13.157.188:46853 with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 02:43:32.225638 21297 heartbeater.cc:347] Connected to a master server at 127.13.157.189:33117
I20260925 02:43:32.225919 21297 heartbeater.cc:464] Registering TS with master...
I20260925 02:43:32.226500 21297 heartbeater.cc:511] Master 127.13.157.189:33117 requested a full tablet report, sending...
I20260925 02:43:32.226622 20741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:32.214533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.188:46853 (local address 127.0.0.1:34848)
0925 02:43:32.214804 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:32.214818 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:32.214900 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:32.222557 (+ 7657us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:32.222606 (+ 49us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:32.222632 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:43:32.222731 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:32.222738 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:32.223778 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:32.223782 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:32.224765 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:32.224776 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:32.226235 (+ 1459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:32.226239 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:32.226331 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:32.226353 (+ 22us) client_negotiation.cc:769] Sending connection context
0925 02:43:32.226426 (+ 73us) client_negotiation.cc:240] Negotiation successful
0925 02:43:32.226498 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":194}
I20260925 02:43:32.226758 21301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:32.208101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.188:46853 (local address 127.13.157.130:33619)
0925 02:43:32.208343 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:32.208353 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:32.208695 (+ 342us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:32.209566 (+ 871us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:32.209569 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:32.209856 (+ 287us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:32.210470 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:32.210480 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:32.211480 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:32.211483 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:32.213023 (+ 1540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:32.213035 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:32.213766 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:32.213770 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:32.213864 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:32.216370 (+ 2506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:32.216384 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:32.217816 (+ 1432us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:43:32.219725 (+ 1909us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:32.219728 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:32.219730 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:32.219950 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:32.220353 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:32.220356 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:32.220358 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:32.220430 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:32.220863 (+ 433us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:43:32.220865 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:43:32.221210 (+ 345us) client_negotiation.cc:769] Sending connection context
0925 02:43:32.226425 (+ 5215us) client_negotiation.cc:240] Negotiation successful
0925 02:43:32.226576 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"mutex_wait_us":254,"spinlock_wait_cycles":46592,"thread_start_us":74,"threads_started":1}
I20260925 02:43:32.225664 21296 heartbeater.cc:347] Connected to a master server at 127.13.157.190:34721
I20260925 02:43:32.227066 21296 heartbeater.cc:464] Registering TS with master...
I20260925 02:43:32.227248 21296 heartbeater.cc:511] Master 127.13.157.190:34721 requested a full tablet report, sending...
I20260925 02:43:32.227332 21295 heartbeater.cc:347] Connected to a master server at 127.13.157.188:46853
I20260925 02:43:32.227375 21161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:32.215078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34848 (local address 127.13.157.188:46853)
0925 02:43:32.221994 (+ 6916us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:32.221998 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:32.222011 (+ 13us) server_negotiation.cc:407] Connection header received
0925 02:43:32.222175 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:32.222179 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:32.222250 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:32.222347 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:43:32.222884 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:32.223652 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:32.225074 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:32.226126 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:32.226235 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:32.226379 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:32.227084 (+ 705us) server_negotiation.cc:299] Negotiation successful
0925 02:43:32.227195 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6857,"thread_start_us":102,"threads_started":1}
I20260925 02:43:32.227420 21295 heartbeater.cc:464] Registering TS with master...
I20260925 02:43:32.227584 21295 heartbeater.cc:511] Master 127.13.157.188:46853 requested a full tablet report, sending...
I20260925 02:43:32.227697 20787 ts_manager.cc:194] Registered new tserver with Master: 03b8c1c27f5b471498f53d8d326b61db (127.13.157.130:36015)
I20260925 02:43:32.227954 20171 ts_manager.cc:194] Re-registered known tserver with Master: 03b8c1c27f5b471498f53d8d326b61db (127.13.157.130:36015)
I20260925 02:43:32.228191 20983 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 82186af4c8cb4282a83c2d2936d226ac [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260925 02:43:32.228643 20171 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.157.130@KRBTEST.COM'} at 127.13.157.130:37209
I20260925 02:43:32.228684 21013 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82186af4c8cb4282a83c2d2936d226ac [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2b4937775ce34bed9ca35afdca0243b0. Latest consensus state: current_term: 1 leader_uuid: "2b4937775ce34bed9ca35afdca0243b0" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "2b4937775ce34bed9ca35afdca0243b0" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 34721 } } peers { permanent_uuid: "1d39673cf6e84fd9a7012f26f96107e0" member_type: VOTER last_known_addr { host: "127.13.157.189" port: 33117 } attrs { promote: false } } peers { permanent_uuid: "82186af4c8cb4282a83c2d2936d226ac" member_type: NON_VOTER last_known_addr { host: "127.13.157.188" port: 46853 } attrs { promote: true } } }
I20260925 02:43:32.228790 21013 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82186af4c8cb4282a83c2d2936d226ac [sys.catalog]: This master's current role is: LEARNER
I20260925 02:43:32.229604 20973 ts_manager.cc:194] Registered new tserver with Master: 03b8c1c27f5b471498f53d8d326b61db (127.13.157.130:36015)
W20260925 02:43:32.339110 21303 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:43:32.339373 21303 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:43:32.339416 21303 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:43:32.343824 21303 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:43:32.343888 21303 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:43:32.343954 21303 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.157.131
I20260925 02:43:32.349444 21303 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:46123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/krb5kdc/oryx_127.13.157.131.keytab
--principal=oryx/127.13.157.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.157.131:33075
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.157.131
--webserver_port=37925
--tserver_master_addrs=127.13.157.190:34721,127.13.157.189:33117,127.13.157.188:46853
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21303
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.157.131
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:43:32.350522 21303 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:43:32.351572 21303 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:43:32.361307 21309 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:43:32.361483 21303 server_base.cc:1034] running on GCE node
W20260925 02:43:32.361354 21308 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:32.361354 21311 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:43:32.362025 21303 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:43:32.362665 21303 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:43:32.363832 21303 hybrid_clock.cc:648] HybridClock initialized: now 1790304212363791 us; error 54 us; skew 500 ppm
Sep 25 02:43:32 dist-test-slave-kznw krb5kdc[20127](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304212, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.157.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:43:32.366568 21303 init.cc:377] Logged in from keytab as oryx/127.13.157.131@KRBTEST.COM (short username oryx)
I20260925 02:43:32.367420 21303 webserver.cc:460] Webserver started at http://127.13.157.131:37925/ using document root <none> and password file <none>
I20260925 02:43:32.367920 21303 fs_manager.cc:359] Metadata directory not provided
I20260925 02:43:32.368031 21303 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:43:32.371882 21303 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260925 02:43:32.373914 21319 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:32.375365 21303 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 02:43:32.375465 21303 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/ts-2/wal
uuid: "323109284d62402a989a4cd4f8ec3c54"
format_stamp: "Formatted at 2026-09-25 02:43:31 on dist-test-slave-kznw"
server_key: "c5d3d6a1ceb9ec3c736634081de6ef39"
server_key_iv: "83ee8d4d33949feb1c59c6024e4fe419"
server_key_version: "encryptionkey@0"
I20260925 02:43:32.376046 21303 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:43:32.388795 21303 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:43:32.391947 21303 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:43:32.392122 21303 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:43:32.392791 21303 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:43:32.393776 21303 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:43:32.393829 21303 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:32.393887 21303 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:43:32.393909 21303 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:32.403330 21303 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.131:33075
I20260925 02:43:32.403379 21434 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.131:33075 every 8 connection(s)
I20260925 02:43:32.404338 21303 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/ts-2/data/info.pb
I20260925 02:43:32.413239 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 21303
Sep 25 02:43:32 dist-test-slave-kznw krb5kdc[20127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304212, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.157.131@KRBTEST.COM for oryx/127.13.157.188@KRBTEST.COM
Sep 25 02:43:32 dist-test-slave-kznw krb5kdc[20127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304212, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.157.131@KRBTEST.COM for oryx/127.13.157.189@KRBTEST.COM
Sep 25 02:43:32 dist-test-slave-kznw krb5kdc[20127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304212, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.157.131@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM
I20260925 02:43:32.423480 20211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:32.406151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.131:36187 (local address 127.13.157.190:34721)
0925 02:43:32.406318 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:32.406323 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:32.407681 (+ 1358us) server_negotiation.cc:407] Connection header received
0925 02:43:32.408815 (+ 1134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:32.408819 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:32.408874 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:32.408964 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:32.410464 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:32.411343 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:32.412495 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:32.412785 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:32.412897 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:32.418713 (+ 5816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:32.418722 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:32.418725 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:43:32.418759 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:32.420491 (+ 1732us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:32.421244 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:32.421249 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:32.421251 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:32.421347 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:32.421780 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:32.421784 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:32.421786 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:32.422293 (+ 507us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:32.422368 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:32.423147 (+ 779us) server_negotiation.cc:299] Negotiation successful
0925 02:43:32.423272 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62,"spinlock_wait_cycles":768}
I20260925 02:43:32.424496 21439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:32.406549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:34721 (local address 127.13.157.131:36187)
0925 02:43:32.407520 (+ 971us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:32.407581 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:32.408559 (+ 978us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:32.409166 (+ 607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:32.409175 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:32.409742 (+ 567us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:32.410331 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:32.410346 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:32.411452 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:32.411455 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:32.412342 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:32.412353 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:32.415080 (+ 2727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:32.415406 (+ 326us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:32.415502 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:32.416199 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:32.416213 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:32.418113 (+ 1900us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:43:32.420638 (+ 2525us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:32.420641 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:32.420643 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:32.420869 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:32.421477 (+ 608us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:32.421480 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:32.421482 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:32.421664 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:32.422469 (+ 805us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:43:32.422476 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:43:32.422817 (+ 341us) client_negotiation.cc:769] Sending connection context
0925 02:43:32.423848 (+ 1031us) client_negotiation.cc:240] Negotiation successful
0925 02:43:32.423992 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":761,"mutex_wait_us":85,"spinlock_wait_cycles":16512,"thread_start_us":629,"threads_started":1}
I20260925 02:43:32.425783 21440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:32.407177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.189:33117 (local address 127.13.157.131:55853)
0925 02:43:32.407520 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:32.407581 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:32.408565 (+ 984us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:32.409538 (+ 973us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:32.409541 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:32.409748 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:32.410340 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:32.410349 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:32.411230 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:32.411232 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:32.412453 (+ 1221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:32.412464 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:32.413129 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:32.413133 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:32.413333 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:32.414010 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:32.414040 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:32.417233 (+ 3193us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:43:32.420992 (+ 3759us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:32.420995 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:32.420997 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:32.421258 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:32.422083 (+ 825us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:32.422086 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:32.422088 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:32.422184 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:32.425241 (+ 3057us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:43:32.425245 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:43:32.425338 (+ 93us) client_negotiation.cc:769] Sending connection context
0925 02:43:32.425453 (+ 115us) client_negotiation.cc:240] Negotiation successful
0925 02:43:32.425590 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"mutex_wait_us":113,"spinlock_wait_cycles":29056,"thread_start_us":100,"threads_started":1}
I20260925 02:43:32.425791 20910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:32.407260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.131:55853 (local address 127.13.157.189:33117)
0925 02:43:32.407388 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:32.407393 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:32.407681 (+ 288us) server_negotiation.cc:407] Connection header received
0925 02:43:32.408815 (+ 1134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:32.408819 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:32.408874 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:32.408960 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:32.410504 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:32.411079 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:32.412593 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:32.412842 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:32.412918 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:32.417365 (+ 4447us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:32.417369 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:32.417372 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:43:32.417399 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:32.420852 (+ 3453us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:32.421858 (+ 1006us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:32.421862 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:32.421863 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:32.421967 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:32.424842 (+ 2875us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:32.424846 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:32.424848 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:32.425097 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:32.425194 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:32.425534 (+ 340us) server_negotiation.cc:299] Negotiation successful
0925 02:43:32.425643 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260925 02:43:32.425880 21446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:32.415250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32850 (local address 127.13.157.190:34721)
0925 02:43:32.415494 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:32.415498 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:32.416453 (+ 955us) server_negotiation.cc:407] Connection header received
0925 02:43:32.416642 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:32.416646 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:32.416701 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:32.416764 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:32.418121 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:32.418960 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:32.420138 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:32.420475 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:32.420550 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:32.422204 (+ 1654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:32.422208 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:32.422211 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:43:32.422268 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:32.424023 (+ 1755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:32.424533 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:32.424536 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:32.424538 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:32.424637 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:32.424941 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:32.424945 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:32.424946 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:32.425154 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:32.425255 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:32.425590 (+ 335us) server_negotiation.cc:299] Negotiation successful
0925 02:43:32.425719 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173,"spinlock_wait_cycles":1408,"thread_start_us":86,"threads_started":1}
I20260925 02:43:32.426024 21436 heartbeater.cc:347] Connected to a master server at 127.13.157.190:34721
I20260925 02:43:32.426275 21436 heartbeater.cc:464] Registering TS with master...
I20260925 02:43:32.426462 21437 heartbeater.cc:347] Connected to a master server at 127.13.157.189:33117
I20260925 02:43:32.426559 21437 heartbeater.cc:464] Registering TS with master...
I20260925 02:43:32.426823 21161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:32.408208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.131:36181 (local address 127.13.157.188:46853)
0925 02:43:32.408343 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:32.408346 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:32.409051 (+ 705us) server_negotiation.cc:407] Connection header received
0925 02:43:32.409205 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:32.409208 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:32.409250 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:32.409310 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:32.410515 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:32.411344 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:32.412679 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:32.413019 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:32.413130 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:32.417422 (+ 4292us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:32.417427 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:32.417430 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:43:32.417461 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:32.419552 (+ 2091us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:32.421397 (+ 1845us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:32.421410 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:32.421412 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:32.421507 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:32.422300 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:32.422304 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:32.422305 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:32.422562 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:32.422660 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:32.426534 (+ 3874us) server_negotiation.cc:299] Negotiation successful
0925 02:43:32.426664 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260925 02:43:32.426854 21436 heartbeater.cc:511] Master 127.13.157.190:34721 requested a full tablet report, sending...
I20260925 02:43:32.424769 21441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:32.408066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.188:46853 (local address 127.13.157.131:36181)
0925 02:43:32.408976 (+ 910us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:32.408988 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:32.409111 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:32.409346 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:32.409349 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:32.409730 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:32.410343 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:32.410349 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:32.411747 (+ 1398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:32.411750 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:32.412566 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:32.412574 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:32.413143 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:32.413147 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:32.413340 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:32.414052 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:32.414067 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:32.416757 (+ 2690us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:43:32.419726 (+ 2969us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:32.419733 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:32.419749 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:32.420129 (+ 380us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:32.421660 (+ 1531us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:32.421663 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:32.421665 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:32.421753 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:32.422785 (+ 1032us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:43:32.422788 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:43:32.422867 (+ 79us) client_negotiation.cc:769] Sending connection context
0925 02:43:32.423063 (+ 196us) client_negotiation.cc:240] Negotiation successful
0925 02:43:32.423410 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":847,"mutex_wait_us":132,"spinlock_wait_cycles":22144,"thread_start_us":73,"threads_started":1}
I20260925 02:43:32.426854 21437 heartbeater.cc:511] Master 127.13.157.189:33117 requested a full tablet report, sending...
I20260925 02:43:32.427713 21435 heartbeater.cc:347] Connected to a master server at 127.13.157.188:46853
I20260925 02:43:32.427783 21435 heartbeater.cc:464] Registering TS with master...
I20260925 02:43:32.427920 21435 heartbeater.cc:511] Master 127.13.157.188:46853 requested a full tablet report, sending...
I20260925 02:43:32.428666 20973 ts_manager.cc:194] Registered new tserver with Master: 323109284d62402a989a4cd4f8ec3c54 (127.13.157.131:33075)
I20260925 02:43:32.428675 20787 ts_manager.cc:194] Registered new tserver with Master: 323109284d62402a989a4cd4f8ec3c54 (127.13.157.131:33075)
I20260925 02:43:32.429955 20172 ts_manager.cc:194] Re-registered known tserver with Master: 323109284d62402a989a4cd4f8ec3c54 (127.13.157.131:33075)
I20260925 02:43:32.430673 20172 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.157.131@KRBTEST.COM'} at 127.13.157.131:36187
I20260925 02:43:32.431617 21448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:32.415984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56760 (local address 127.13.157.189:33117)
0925 02:43:32.416214 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:32.416217 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:32.417999 (+ 1782us) server_negotiation.cc:407] Connection header received
0925 02:43:32.418166 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:32.418169 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:32.418212 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:32.418267 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:32.419167 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:32.419974 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:32.421034 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:32.421390 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:32.421465 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:32.423233 (+ 1768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:32.423238 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:32.423240 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:43:32.423267 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:32.424914 (+ 1647us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:32.430097 (+ 5183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:32.430101 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:32.430103 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:32.430209 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:32.430560 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:32.430564 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:32.430566 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:32.430786 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:32.430923 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:32.431283 (+ 360us) server_negotiation.cc:299] Negotiation successful
0925 02:43:32.431410 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":82,"threads_started":1}
I20260925 02:43:32.429881 20171 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32850:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 02:43:32.433801 20171 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 02:43:32.435691 21302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:32.415778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34850 (local address 127.13.157.188:46853)
0925 02:43:32.415910 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:32.415913 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:32.417010 (+ 1097us) server_negotiation.cc:407] Connection header received
0925 02:43:32.417167 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:32.417171 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:32.417221 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:32.417278 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:32.427773 (+ 10495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:32.428471 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:32.430336 (+ 1865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:32.430685 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:32.430778 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:32.432618 (+ 1840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:32.432622 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:32.432624 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:43:32.432652 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:32.434081 (+ 1429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:32.434480 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:32.434484 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:32.434486 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:32.434576 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:32.434833 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:32.434837 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:32.434839 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:32.435083 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:32.435213 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:32.435463 (+ 250us) server_negotiation.cc:299] Negotiation successful
0925 02:43:32.435570 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260925 02:43:32.436452 20744 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b4937775ce34bed9ca35afdca0243b0 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2b4937775ce34bed9ca35afdca0243b0" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "2b4937775ce34bed9ca35afdca0243b0" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 34721 } } peers { permanent_uuid: "1d39673cf6e84fd9a7012f26f96107e0" member_type: VOTER last_known_addr { host: "127.13.157.189" port: 33117 } attrs { promote: false } } peers { permanent_uuid: "82186af4c8cb4282a83c2d2936d226ac" member_type: NON_VOTER last_known_addr { host: "127.13.157.188" port: 46853 } attrs { promote: true } } }
I20260925 02:43:32.436596 20744 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b4937775ce34bed9ca35afdca0243b0 [sys.catalog]: This master's current role is: LEADER
I20260925 02:43:32.439854 21453 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 2b4937775ce34bed9ca35afdca0243b0: attempting to promote NON_VOTER 82186af4c8cb4282a83c2d2936d226ac to VOTER
I20260925 02:43:32.440219 21453 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2b4937775ce34bed9ca35afdca0243b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "2b4937775ce34bed9ca35afdca0243b0" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 34721 } } peers { permanent_uuid: "1d39673cf6e84fd9a7012f26f96107e0" member_type: VOTER last_known_addr { host: "127.13.157.189" port: 33117 } attrs { promote: false } } peers { permanent_uuid: "82186af4c8cb4282a83c2d2936d226ac" member_type: VOTER last_known_addr { host: "127.13.157.188" port: 46853 } attrs { promote: false } }
I20260925 02:43:32.441310 20795 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1d39673cf6e84fd9a7012f26f96107e0 [term 1 FOLLOWER]: Refusing update from remote peer 2b4937775ce34bed9ca35afdca0243b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260925 02:43:32.441526 20983 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 82186af4c8cb4282a83c2d2936d226ac [term 1 LEARNER]: Refusing update from remote peer 2b4937775ce34bed9ca35afdca0243b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260925 02:43:32.441640 20744 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2b4937775ce34bed9ca35afdca0243b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d39673cf6e84fd9a7012f26f96107e0" member_type: VOTER last_known_addr { host: "127.13.157.189" port: 33117 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260925 02:43:32.441862 21452 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2b4937775ce34bed9ca35afdca0243b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "82186af4c8cb4282a83c2d2936d226ac" member_type: VOTER last_known_addr { host: "127.13.157.188" port: 46853 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260925 02:43:32.443041 20744 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 2b4937775ce34bed9ca35afdca0243b0 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 82186af4c8cb4282a83c2d2936d226ac (127.13.157.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "2b4937775ce34bed9ca35afdca0243b0" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 34721 } } peers { permanent_uuid: "1d39673cf6e84fd9a7012f26f96107e0" member_type: VOTER last_known_addr { host: "127.13.157.189" port: 33117 } attrs { promote: false } } peers { permanent_uuid: "82186af4c8cb4282a83c2d2936d226ac" member_type: VOTER last_known_addr { host: "127.13.157.188" port: 46853 } attrs { promote: false } } }
I20260925 02:43:32.443706 20795 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1d39673cf6e84fd9a7012f26f96107e0 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 82186af4c8cb4282a83c2d2936d226ac (127.13.157.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "2b4937775ce34bed9ca35afdca0243b0" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 34721 } } peers { permanent_uuid: "1d39673cf6e84fd9a7012f26f96107e0" member_type: VOTER last_known_addr { host: "127.13.157.189" port: 33117 } attrs { promote: false } } peers { permanent_uuid: "82186af4c8cb4282a83c2d2936d226ac" member_type: VOTER last_known_addr { host: "127.13.157.188" port: 46853 } attrs { promote: false } } }
I20260925 02:43:32.444437 20744 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b4937775ce34bed9ca35afdca0243b0 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "2b4937775ce34bed9ca35afdca0243b0" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "2b4937775ce34bed9ca35afdca0243b0" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 34721 } } peers { permanent_uuid: "1d39673cf6e84fd9a7012f26f96107e0" member_type: VOTER last_known_addr { host: "127.13.157.189" port: 33117 } attrs { promote: false } } peers { permanent_uuid: "82186af4c8cb4282a83c2d2936d226ac" member_type: VOTER last_known_addr { host: "127.13.157.188" port: 46853 } attrs { promote: false } } }
I20260925 02:43:32.444540 20744 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b4937775ce34bed9ca35afdca0243b0 [sys.catalog]: This master's current role is: LEADER
I20260925 02:43:32.444648 20744 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b4937775ce34bed9ca35afdca0243b0 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2b4937775ce34bed9ca35afdca0243b0" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "2b4937775ce34bed9ca35afdca0243b0" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 34721 } } peers { permanent_uuid: "1d39673cf6e84fd9a7012f26f96107e0" member_type: VOTER last_known_addr { host: "127.13.157.189" port: 33117 } attrs { promote: false } } peers { permanent_uuid: "82186af4c8cb4282a83c2d2936d226ac" member_type: VOTER last_known_addr { host: "127.13.157.188" port: 46853 } attrs { promote: false } } }
I20260925 02:43:32.444720 20744 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b4937775ce34bed9ca35afdca0243b0 [sys.catalog]: This master's current role is: LEADER
I20260925 02:43:32.444788 21453 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b4937775ce34bed9ca35afdca0243b0 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2b4937775ce34bed9ca35afdca0243b0" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "2b4937775ce34bed9ca35afdca0243b0" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 34721 } } peers { permanent_uuid: "1d39673cf6e84fd9a7012f26f96107e0" member_type: VOTER last_known_addr { host: "127.13.157.189" port: 33117 } attrs { promote: false } } peers { permanent_uuid: "82186af4c8cb4282a83c2d2936d226ac" member_type: VOTER last_known_addr { host: "127.13.157.188" port: 46853 } attrs { promote: false } } }
I20260925 02:43:32.444890 21453 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b4937775ce34bed9ca35afdca0243b0 [sys.catalog]: This master's current role is: LEADER
I20260925 02:43:32.445218 21461 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d39673cf6e84fd9a7012f26f96107e0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2b4937775ce34bed9ca35afdca0243b0" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "2b4937775ce34bed9ca35afdca0243b0" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 34721 } } peers { permanent_uuid: "1d39673cf6e84fd9a7012f26f96107e0" member_type: VOTER last_known_addr { host: "127.13.157.189" port: 33117 } attrs { promote: false } } peers { permanent_uuid: "82186af4c8cb4282a83c2d2936d226ac" member_type: VOTER last_known_addr { host: "127.13.157.188" port: 46853 } attrs { promote: false } } }
I20260925 02:43:32.445338 21461 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d39673cf6e84fd9a7012f26f96107e0 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 02:43:32.445494 20983 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 82186af4c8cb4282a83c2d2936d226ac [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 82186af4c8cb4282a83c2d2936d226ac (127.13.157.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "2b4937775ce34bed9ca35afdca0243b0" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 34721 } } peers { permanent_uuid: "1d39673cf6e84fd9a7012f26f96107e0" member_type: VOTER last_known_addr { host: "127.13.157.189" port: 33117 } attrs { promote: false } } peers { permanent_uuid: "82186af4c8cb4282a83c2d2936d226ac" member_type: VOTER last_known_addr { host: "127.13.157.188" port: 46853 } attrs { promote: false } } }
I20260925 02:43:32.447487 21013 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82186af4c8cb4282a83c2d2936d226ac [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2b4937775ce34bed9ca35afdca0243b0" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "2b4937775ce34bed9ca35afdca0243b0" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 34721 } } peers { permanent_uuid: "1d39673cf6e84fd9a7012f26f96107e0" member_type: VOTER last_known_addr { host: "127.13.157.189" port: 33117 } attrs { promote: false } } peers { permanent_uuid: "82186af4c8cb4282a83c2d2936d226ac" member_type: VOTER last_known_addr { host: "127.13.157.188" port: 46853 } attrs { promote: false } } }
I20260925 02:43:32.447595 21013 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82186af4c8cb4282a83c2d2936d226ac [sys.catalog]: This master's current role is: FOLLOWER
I20260925 02:43:32.455603 21464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:32.448964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.130:36015 (local address 127.0.0.1:60118)
0925 02:43:32.450033 (+ 1069us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:32.450046 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:32.450162 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:32.451359 (+ 1197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:32.451362 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:32.451382 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:43:32.451487 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:32.451493 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:32.452685 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:32.452689 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:32.453949 (+ 1260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:32.453993 (+ 44us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:32.455215 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:32.455219 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:32.455307 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:32.455330 (+ 23us) client_negotiation.cc:769] Sending connection context
0925 02:43:32.455387 (+ 57us) client_negotiation.cc:240] Negotiation successful
0925 02:43:32.455453 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1008,"mutex_wait_us":21,"thread_start_us":169,"threads_started":1}
I20260925 02:43:32.455819 21463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:32.449425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.129:46073 (local address 127.0.0.1:51240)
0925 02:43:32.451088 (+ 1663us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:32.451098 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:32.451186 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:32.451630 (+ 444us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:32.451633 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:32.451652 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:43:32.451736 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:32.451743 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:32.453178 (+ 1435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:32.453181 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:32.454330 (+ 1149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:32.454339 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:32.455486 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:32.455489 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:32.455570 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:32.455591 (+ 21us) client_negotiation.cc:769] Sending connection context
0925 02:43:32.455656 (+ 65us) client_negotiation.cc:240] Negotiation successful
0925 02:43:32.455720 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1609,"mutex_wait_us":43,"thread_start_us":135,"threads_started":1}
I20260925 02:43:32.455966 21466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:32.449329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60118 (local address 127.13.157.130:36015)
0925 02:43:32.450838 (+ 1509us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:32.450868 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:32.450889 (+ 21us) server_negotiation.cc:407] Connection header received
0925 02:43:32.451005 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:32.451011 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:32.451201 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:32.451376 (+ 175us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:43:32.451578 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:32.452555 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:32.454133 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:32.455096 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:32.455216 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:32.455364 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:32.455421 (+ 57us) server_negotiation.cc:299] Negotiation successful
0925 02:43:32.455519 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1420,"thread_start_us":221,"threads_started":1}
I20260925 02:43:32.456427 21467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:32.449858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51240 (local address 127.13.157.129:46073)
0925 02:43:32.450205 (+ 347us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:32.450216 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:32.451183 (+ 967us) server_negotiation.cc:407] Connection header received
0925 02:43:32.451289 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:32.451295 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:32.451505 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:32.451660 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:43:32.451826 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:32.453024 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:32.454441 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:32.455177 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:32.455293 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:32.455418 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:32.456136 (+ 718us) server_negotiation.cc:299] Negotiation successful
0925 02:43:32.456236 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":255,"thread_start_us":158,"threads_started":1}
I20260925 02:43:32.457264 20741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:32.448416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.131:33075 (local address 127.0.0.1:53490)
0925 02:43:32.448585 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:32.448600 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:32.448675 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:32.451828 (+ 3153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:32.451831 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:32.451849 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:43:32.451925 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:32.451931 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:32.453595 (+ 1664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:32.453599 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:32.454804 (+ 1205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:32.454820 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:32.456888 (+ 2068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:32.456891 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:32.456984 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:32.457005 (+ 21us) client_negotiation.cc:769] Sending connection context
0925 02:43:32.457082 (+ 77us) client_negotiation.cc:240] Negotiation successful
0925 02:43:32.457149 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94,"mutex_wait_us":53}
I20260925 02:43:32.457351 21462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:32.448980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53490 (local address 127.13.157.131:33075)
0925 02:43:32.451428 (+ 2448us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:32.451438 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:32.451459 (+ 21us) server_negotiation.cc:407] Connection header received
0925 02:43:32.451527 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:32.451531 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:32.451691 (+ 160us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:32.452160 (+ 469us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:43:32.452302 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:32.453402 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:32.454980 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:32.456747 (+ 1767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:32.456896 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:32.457010 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:32.457107 (+ 97us) server_negotiation.cc:299] Negotiation successful
0925 02:43:32.457190 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2351,"thread_start_us":131,"threads_started":1}
I20260925 02:43:32.459060 21228 tablet_service.cc:1467] Processing CreateTablet for tablet 13843a3402bd4385a9a6cbaa5d11c96e (DEFAULT_TABLE table=test-table [id=1fa1dbfc78a2460380ff8cfd98209350]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:43:32.459107 21090 tablet_service.cc:1467] Processing CreateTablet for tablet 13843a3402bd4385a9a6cbaa5d11c96e (DEFAULT_TABLE table=test-table [id=1fa1dbfc78a2460380ff8cfd98209350]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:43:32.460163 21090 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 13843a3402bd4385a9a6cbaa5d11c96e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:43:32.460163 21228 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 13843a3402bd4385a9a6cbaa5d11c96e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:43:32.459776 21368 tablet_service.cc:1467] Processing CreateTablet for tablet 13843a3402bd4385a9a6cbaa5d11c96e (DEFAULT_TABLE table=test-table [id=1fa1dbfc78a2460380ff8cfd98209350]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:43:32.460793 21368 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 13843a3402bd4385a9a6cbaa5d11c96e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:43:32.467659 21468 tablet_bootstrap.cc:492] T 13843a3402bd4385a9a6cbaa5d11c96e P 323109284d62402a989a4cd4f8ec3c54: Bootstrap starting.
I20260925 02:43:32.468559 21470 tablet_bootstrap.cc:492] T 13843a3402bd4385a9a6cbaa5d11c96e P 03b8c1c27f5b471498f53d8d326b61db: Bootstrap starting.
I20260925 02:43:32.468830 21469 tablet_bootstrap.cc:492] T 13843a3402bd4385a9a6cbaa5d11c96e P 15b827f1b5534fafa26e3142b9e9d192: Bootstrap starting.
I20260925 02:43:32.471643 21468 tablet_bootstrap.cc:654] T 13843a3402bd4385a9a6cbaa5d11c96e P 323109284d62402a989a4cd4f8ec3c54: Neither blocks nor log segments found. Creating new log.
I20260925 02:43:32.472436 21469 tablet_bootstrap.cc:654] T 13843a3402bd4385a9a6cbaa5d11c96e P 15b827f1b5534fafa26e3142b9e9d192: Neither blocks nor log segments found. Creating new log.
I20260925 02:43:32.472653 21470 tablet_bootstrap.cc:654] T 13843a3402bd4385a9a6cbaa5d11c96e P 03b8c1c27f5b471498f53d8d326b61db: Neither blocks nor log segments found. Creating new log.
I20260925 02:43:32.473253 21468 log.cc:824] T 13843a3402bd4385a9a6cbaa5d11c96e P 323109284d62402a989a4cd4f8ec3c54: Log is configured to *not* fsync() on all Append() calls
I20260925 02:43:32.473510 21470 log.cc:824] T 13843a3402bd4385a9a6cbaa5d11c96e P 03b8c1c27f5b471498f53d8d326b61db: Log is configured to *not* fsync() on all Append() calls
I20260925 02:43:32.473878 21469 log.cc:824] T 13843a3402bd4385a9a6cbaa5d11c96e P 15b827f1b5534fafa26e3142b9e9d192: Log is configured to *not* fsync() on all Append() calls
I20260925 02:43:32.475546 21470 tablet_bootstrap.cc:492] T 13843a3402bd4385a9a6cbaa5d11c96e P 03b8c1c27f5b471498f53d8d326b61db: No bootstrap required, opened a new log
I20260925 02:43:32.475546 21468 tablet_bootstrap.cc:492] T 13843a3402bd4385a9a6cbaa5d11c96e P 323109284d62402a989a4cd4f8ec3c54: No bootstrap required, opened a new log
I20260925 02:43:32.475728 21470 ts_tablet_manager.cc:1397] T 13843a3402bd4385a9a6cbaa5d11c96e P 03b8c1c27f5b471498f53d8d326b61db: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.004s
I20260925 02:43:32.475728 21468 ts_tablet_manager.cc:1397] T 13843a3402bd4385a9a6cbaa5d11c96e P 323109284d62402a989a4cd4f8ec3c54: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260925 02:43:32.480482 21470 raft_consensus.cc:348] T 13843a3402bd4385a9a6cbaa5d11c96e P 03b8c1c27f5b471498f53d8d326b61db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "323109284d62402a989a4cd4f8ec3c54" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 33075 } } peers { permanent_uuid: "15b827f1b5534fafa26e3142b9e9d192" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 46073 } } peers { permanent_uuid: "03b8c1c27f5b471498f53d8d326b61db" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 36015 } }
I20260925 02:43:32.480479 21468 raft_consensus.cc:348] T 13843a3402bd4385a9a6cbaa5d11c96e P 323109284d62402a989a4cd4f8ec3c54 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "323109284d62402a989a4cd4f8ec3c54" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 33075 } } peers { permanent_uuid: "15b827f1b5534fafa26e3142b9e9d192" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 46073 } } peers { permanent_uuid: "03b8c1c27f5b471498f53d8d326b61db" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 36015 } }
I20260925 02:43:32.480760 21468 raft_consensus.cc:374] T 13843a3402bd4385a9a6cbaa5d11c96e P 323109284d62402a989a4cd4f8ec3c54 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:43:32.480760 21470 raft_consensus.cc:374] T 13843a3402bd4385a9a6cbaa5d11c96e P 03b8c1c27f5b471498f53d8d326b61db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:43:32.480818 21468 raft_consensus.cc:729] T 13843a3402bd4385a9a6cbaa5d11c96e P 323109284d62402a989a4cd4f8ec3c54 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 323109284d62402a989a4cd4f8ec3c54, State: Initialized, Role: FOLLOWER
I20260925 02:43:32.480825 21470 raft_consensus.cc:729] T 13843a3402bd4385a9a6cbaa5d11c96e P 03b8c1c27f5b471498f53d8d326b61db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03b8c1c27f5b471498f53d8d326b61db, State: Initialized, Role: FOLLOWER
I20260925 02:43:32.481290 21468 consensus_queue.cc:260] T 13843a3402bd4385a9a6cbaa5d11c96e P 323109284d62402a989a4cd4f8ec3c54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "323109284d62402a989a4cd4f8ec3c54" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 33075 } } peers { permanent_uuid: "15b827f1b5534fafa26e3142b9e9d192" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 46073 } } peers { permanent_uuid: "03b8c1c27f5b471498f53d8d326b61db" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 36015 } }
I20260925 02:43:32.481290 21470 consensus_queue.cc:260] T 13843a3402bd4385a9a6cbaa5d11c96e P 03b8c1c27f5b471498f53d8d326b61db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "323109284d62402a989a4cd4f8ec3c54" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 33075 } } peers { permanent_uuid: "15b827f1b5534fafa26e3142b9e9d192" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 46073 } } peers { permanent_uuid: "03b8c1c27f5b471498f53d8d326b61db" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 36015 } }
I20260925 02:43:32.482349 21468 ts_tablet_manager.cc:1428] T 13843a3402bd4385a9a6cbaa5d11c96e P 323109284d62402a989a4cd4f8ec3c54: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260925 02:43:32.482349 21470 ts_tablet_manager.cc:1428] T 13843a3402bd4385a9a6cbaa5d11c96e P 03b8c1c27f5b471498f53d8d326b61db: Time spent starting tablet: real 0.007s user 0.003s sys 0.004s
I20260925 02:43:32.482604 21296 heartbeater.cc:503] Master 127.13.157.190:34721 was elected leader, sending a full tablet report...
I20260925 02:43:32.483640 21436 heartbeater.cc:503] Master 127.13.157.190:34721 was elected leader, sending a full tablet report...
I20260925 02:43:32.485994 21469 tablet_bootstrap.cc:492] T 13843a3402bd4385a9a6cbaa5d11c96e P 15b827f1b5534fafa26e3142b9e9d192: No bootstrap required, opened a new log
I20260925 02:43:32.486193 21469 ts_tablet_manager.cc:1397] T 13843a3402bd4385a9a6cbaa5d11c96e P 15b827f1b5534fafa26e3142b9e9d192: Time spent bootstrapping tablet: real 0.018s user 0.006s sys 0.000s
I20260925 02:43:32.490255 21469 raft_consensus.cc:348] T 13843a3402bd4385a9a6cbaa5d11c96e P 15b827f1b5534fafa26e3142b9e9d192 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "323109284d62402a989a4cd4f8ec3c54" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 33075 } } peers { permanent_uuid: "15b827f1b5534fafa26e3142b9e9d192" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 46073 } } peers { permanent_uuid: "03b8c1c27f5b471498f53d8d326b61db" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 36015 } }
I20260925 02:43:32.490458 21469 raft_consensus.cc:374] T 13843a3402bd4385a9a6cbaa5d11c96e P 15b827f1b5534fafa26e3142b9e9d192 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:43:32.490500 21469 raft_consensus.cc:729] T 13843a3402bd4385a9a6cbaa5d11c96e P 15b827f1b5534fafa26e3142b9e9d192 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15b827f1b5534fafa26e3142b9e9d192, State: Initialized, Role: FOLLOWER
I20260925 02:43:32.490888 21469 consensus_queue.cc:260] T 13843a3402bd4385a9a6cbaa5d11c96e P 15b827f1b5534fafa26e3142b9e9d192 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "323109284d62402a989a4cd4f8ec3c54" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 33075 } } peers { permanent_uuid: "15b827f1b5534fafa26e3142b9e9d192" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 46073 } } peers { permanent_uuid: "03b8c1c27f5b471498f53d8d326b61db" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 36015 } }
I20260925 02:43:32.491539 21157 heartbeater.cc:503] Master 127.13.157.190:34721 was elected leader, sending a full tablet report...
I20260925 02:43:32.491858 21469 ts_tablet_manager.cc:1428] T 13843a3402bd4385a9a6cbaa5d11c96e P 15b827f1b5534fafa26e3142b9e9d192: Time spent starting tablet: real 0.006s user 0.000s sys 0.005s
I20260925 02:43:32.495925 21474 raft_consensus.cc:482] T 13843a3402bd4385a9a6cbaa5d11c96e P 03b8c1c27f5b471498f53d8d326b61db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 02:43:32.496086 21474 raft_consensus.cc:504] T 13843a3402bd4385a9a6cbaa5d11c96e P 03b8c1c27f5b471498f53d8d326b61db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "323109284d62402a989a4cd4f8ec3c54" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 33075 } } peers { permanent_uuid: "15b827f1b5534fafa26e3142b9e9d192" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 46073 } } peers { permanent_uuid: "03b8c1c27f5b471498f53d8d326b61db" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 36015 } }
I20260925 02:43:32.497161 21474 leader_election.cc:288] T 13843a3402bd4385a9a6cbaa5d11c96e P 03b8c1c27f5b471498f53d8d326b61db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 323109284d62402a989a4cd4f8ec3c54 (127.13.157.131:33075), 15b827f1b5534fafa26e3142b9e9d192 (127.13.157.129:46073)
I20260925 02:43:32.500622 21462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:32.497592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.130:49777 (local address 127.13.157.131:33075)
0925 02:43:32.497723 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:32.497727 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:32.497741 (+ 14us) server_negotiation.cc:407] Connection header received
0925 02:43:32.497786 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:32.497789 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:32.497831 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:32.497918 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:43:32.498163 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:32.498667 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:32.499733 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:32.500116 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:32.500216 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:32.500269 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:32.500435 (+ 166us) server_negotiation.cc:299] Negotiation successful
0925 02:43:32.500508 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260925 02:43:32.500638 21301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:32.497335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.131:33075 (local address 127.13.157.130:49777)
0925 02:43:32.497447 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:32.497462 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:32.497574 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:32.497931 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:32.497934 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:32.497950 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:43:32.498065 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:32.498071 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:32.498762 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:32.498765 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:32.499604 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:32.499613 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:32.500217 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:32.500220 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:32.500323 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:32.500346 (+ 23us) client_negotiation.cc:769] Sending connection context
0925 02:43:32.500411 (+ 65us) client_negotiation.cc:240] Negotiation successful
0925 02:43:32.500482 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32,"mutex_wait_us":1}
I20260925 02:43:32.500691 21467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:32.497713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.130:47289 (local address 127.13.157.129:46073)
0925 02:43:32.497838 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:32.497842 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:32.497852 (+ 10us) server_negotiation.cc:407] Connection header received
0925 02:43:32.497910 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:32.497913 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:32.497954 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:32.498053 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:43:32.498286 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:32.498797 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:32.499816 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:32.500213 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:32.500314 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:32.500361 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:32.500518 (+ 157us) server_negotiation.cc:299] Negotiation successful
0925 02:43:32.500579 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260925 02:43:32.501117 21300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:32.497428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.129:46073 (local address 127.13.157.130:47289)
0925 02:43:32.497539 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:32.497553 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:32.497629 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:32.498072 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:32.498075 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:32.498091 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:43:32.498192 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:32.498198 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:32.498894 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:32.498897 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:32.499704 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:32.499713 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:32.500314 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:32.500318 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:32.500411 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:32.500432 (+ 21us) client_negotiation.cc:769] Sending connection context
0925 02:43:32.500494 (+ 62us) client_negotiation.cc:240] Negotiation successful
0925 02:43:32.500570 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42,"mutex_wait_us":18}
I20260925 02:43:32.501385 21389 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13843a3402bd4385a9a6cbaa5d11c96e" candidate_uuid: "03b8c1c27f5b471498f53d8d326b61db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "323109284d62402a989a4cd4f8ec3c54" is_pre_election: true
I20260925 02:43:32.501675 21389 raft_consensus.cc:2454] T 13843a3402bd4385a9a6cbaa5d11c96e P 323109284d62402a989a4cd4f8ec3c54 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 03b8c1c27f5b471498f53d8d326b61db in term 0.
I20260925 02:43:32.501659 21110 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13843a3402bd4385a9a6cbaa5d11c96e" candidate_uuid: "03b8c1c27f5b471498f53d8d326b61db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15b827f1b5534fafa26e3142b9e9d192" is_pre_election: true
I20260925 02:43:32.501886 21110 raft_consensus.cc:2454] T 13843a3402bd4385a9a6cbaa5d11c96e P 15b827f1b5534fafa26e3142b9e9d192 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 03b8c1c27f5b471498f53d8d326b61db in term 0.
I20260925 02:43:32.502090 21182 leader_election.cc:302] T 13843a3402bd4385a9a6cbaa5d11c96e P 03b8c1c27f5b471498f53d8d326b61db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03b8c1c27f5b471498f53d8d326b61db, 323109284d62402a989a4cd4f8ec3c54; no voters:
I20260925 02:43:32.502344 21474 raft_consensus.cc:2781] T 13843a3402bd4385a9a6cbaa5d11c96e P 03b8c1c27f5b471498f53d8d326b61db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 02:43:32.502416 21474 raft_consensus.cc:482] T 13843a3402bd4385a9a6cbaa5d11c96e P 03b8c1c27f5b471498f53d8d326b61db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 02:43:32.502446 21474 raft_consensus.cc:3037] T 13843a3402bd4385a9a6cbaa5d11c96e P 03b8c1c27f5b471498f53d8d326b61db [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:43:32.503545 21474 raft_consensus.cc:504] T 13843a3402bd4385a9a6cbaa5d11c96e P 03b8c1c27f5b471498f53d8d326b61db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "323109284d62402a989a4cd4f8ec3c54" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 33075 } } peers { permanent_uuid: "15b827f1b5534fafa26e3142b9e9d192" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 46073 } } peers { permanent_uuid: "03b8c1c27f5b471498f53d8d326b61db" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 36015 } }
I20260925 02:43:32.503854 21474 leader_election.cc:288] T 13843a3402bd4385a9a6cbaa5d11c96e P 03b8c1c27f5b471498f53d8d326b61db [CANDIDATE]: Term 1 election: Requested vote from peers 323109284d62402a989a4cd4f8ec3c54 (127.13.157.131:33075), 15b827f1b5534fafa26e3142b9e9d192 (127.13.157.129:46073)
I20260925 02:43:32.504165 21389 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13843a3402bd4385a9a6cbaa5d11c96e" candidate_uuid: "03b8c1c27f5b471498f53d8d326b61db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "323109284d62402a989a4cd4f8ec3c54"
I20260925 02:43:32.504192 21110 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13843a3402bd4385a9a6cbaa5d11c96e" candidate_uuid: "03b8c1c27f5b471498f53d8d326b61db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15b827f1b5534fafa26e3142b9e9d192"
I20260925 02:43:32.504287 21389 raft_consensus.cc:3037] T 13843a3402bd4385a9a6cbaa5d11c96e P 323109284d62402a989a4cd4f8ec3c54 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:43:32.504285 21110 raft_consensus.cc:3037] T 13843a3402bd4385a9a6cbaa5d11c96e P 15b827f1b5534fafa26e3142b9e9d192 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:43:32.505504 21389 raft_consensus.cc:2454] T 13843a3402bd4385a9a6cbaa5d11c96e P 323109284d62402a989a4cd4f8ec3c54 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 03b8c1c27f5b471498f53d8d326b61db in term 1.
I20260925 02:43:32.505504 21110 raft_consensus.cc:2454] T 13843a3402bd4385a9a6cbaa5d11c96e P 15b827f1b5534fafa26e3142b9e9d192 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 03b8c1c27f5b471498f53d8d326b61db in term 1.
I20260925 02:43:32.505789 21182 leader_election.cc:302] T 13843a3402bd4385a9a6cbaa5d11c96e P 03b8c1c27f5b471498f53d8d326b61db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 03b8c1c27f5b471498f53d8d326b61db, 15b827f1b5534fafa26e3142b9e9d192, 323109284d62402a989a4cd4f8ec3c54; no voters:
I20260925 02:43:32.505990 21474 raft_consensus.cc:2781] T 13843a3402bd4385a9a6cbaa5d11c96e P 03b8c1c27f5b471498f53d8d326b61db [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:43:32.506173 21474 raft_consensus.cc:686] T 13843a3402bd4385a9a6cbaa5d11c96e P 03b8c1c27f5b471498f53d8d326b61db [term 1 LEADER]: Becoming Leader. State: Replica: 03b8c1c27f5b471498f53d8d326b61db, State: Running, Role: LEADER
I20260925 02:43:32.506486 21474 consensus_queue.cc:237] T 13843a3402bd4385a9a6cbaa5d11c96e P 03b8c1c27f5b471498f53d8d326b61db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "323109284d62402a989a4cd4f8ec3c54" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 33075 } } peers { permanent_uuid: "15b827f1b5534fafa26e3142b9e9d192" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 46073 } } peers { permanent_uuid: "03b8c1c27f5b471498f53d8d326b61db" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 36015 } }
I20260925 02:43:32.509796 20171 catalog_manager.cc:5495] T 13843a3402bd4385a9a6cbaa5d11c96e P 03b8c1c27f5b471498f53d8d326b61db reported cstate change: term changed from 0 to 1, leader changed from <none> to 03b8c1c27f5b471498f53d8d326b61db (127.13.157.130). New cstate: current_term: 1 leader_uuid: "03b8c1c27f5b471498f53d8d326b61db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "323109284d62402a989a4cd4f8ec3c54" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "15b827f1b5534fafa26e3142b9e9d192" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 46073 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "03b8c1c27f5b471498f53d8d326b61db" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 36015 } health_report { overall_health: HEALTHY } } }
W20260925 02:43:32.520157 21159 tablet.cc:2369] T 13843a3402bd4385a9a6cbaa5d11c96e P 15b827f1b5534fafa26e3142b9e9d192: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 02:43:32.533594 21466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:32.530295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60120 (local address 127.13.157.130:36015)
0925 02:43:32.530426 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:32.530431 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:32.530445 (+ 14us) server_negotiation.cc:407] Connection header received
0925 02:43:32.530557 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:32.530560 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:32.530613 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:32.530701 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 02:43:32.530944 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:32.531435 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:32.532251 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:32.532461 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:32.532551 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:32.532823 (+ 272us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 02:43:32.533230 (+ 407us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 02:43:32.533333 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:32.533402 (+ 69us) server_negotiation.cc:299] Negotiation successful
0925 02:43:32.533455 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260925 02:43:32.539753 21110 raft_consensus.cc:1261] T 13843a3402bd4385a9a6cbaa5d11c96e P 15b827f1b5534fafa26e3142b9e9d192 [term 1 FOLLOWER]: Refusing update from remote peer 03b8c1c27f5b471498f53d8d326b61db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:43:32.539803 21389 raft_consensus.cc:1261] T 13843a3402bd4385a9a6cbaa5d11c96e P 323109284d62402a989a4cd4f8ec3c54 [term 1 FOLLOWER]: Refusing update from remote peer 03b8c1c27f5b471498f53d8d326b61db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:43:32.540465 21477 consensus_queue.cc:1035] T 13843a3402bd4385a9a6cbaa5d11c96e P 03b8c1c27f5b471498f53d8d326b61db [LEADER]: Connected to new peer: Peer: permanent_uuid: "15b827f1b5534fafa26e3142b9e9d192" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 46073 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:43:32.540652 21474 consensus_queue.cc:1035] T 13843a3402bd4385a9a6cbaa5d11c96e P 03b8c1c27f5b471498f53d8d326b61db [LEADER]: Connected to new peer: Peer: permanent_uuid: "323109284d62402a989a4cd4f8ec3c54" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:43:32.548794 21480 mvcc.cc:205] Tried to move back new op lower bound from 7333086054555226112 to 7333086054429274112. Current Snapshot: MvccSnapshot[applied={T|T < 7333086054555226112}]
I20260925 02:43:32.550077 21481 mvcc.cc:205] Tried to move back new op lower bound from 7333086054555226112 to 7333086054429274112. Current Snapshot: MvccSnapshot[applied={T|T < 7333086054555226112}]
I20260925 02:43:32.551752 21482 mvcc.cc:205] Tried to move back new op lower bound from 7333086054555226112 to 7333086054429274112. Current Snapshot: MvccSnapshot[applied={T|T < 7333086054555226112}]
I20260925 02:43:32.561748 20171 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32850:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 02:43:32.561895 20171 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32850:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 02:43:32.565845 20171 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2b4937775ce34bed9ca35afdca0243b0: Sending DeleteTablet for 3 replicas of tablet 13843a3402bd4385a9a6cbaa5d11c96e
I20260925 02:43:32.566624 21090 tablet_service.cc:1514] Processing DeleteTablet for tablet 13843a3402bd4385a9a6cbaa5d11c96e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:43:32 UTC) from {username='oryx', principal='oryx/127.13.157.190@KRBTEST.COM'} at 127.0.0.1:51240
I20260925 02:43:32.566704 21228 tablet_service.cc:1514] Processing DeleteTablet for tablet 13843a3402bd4385a9a6cbaa5d11c96e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:43:32 UTC) from {username='oryx', principal='oryx/127.13.157.190@KRBTEST.COM'} at 127.0.0.1:60118
I20260925 02:43:32.566741 21368 tablet_service.cc:1514] Processing DeleteTablet for tablet 13843a3402bd4385a9a6cbaa5d11c96e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:43:32 UTC) from {username='oryx', principal='oryx/127.13.157.190@KRBTEST.COM'} at 127.0.0.1:53490
I20260925 02:43:32.567152 21490 tablet_replica.cc:330] T 13843a3402bd4385a9a6cbaa5d11c96e P 15b827f1b5534fafa26e3142b9e9d192: stopping tablet replica
I20260925 02:43:32.567339 21492 tablet_replica.cc:330] T 13843a3402bd4385a9a6cbaa5d11c96e P 323109284d62402a989a4cd4f8ec3c54: stopping tablet replica
I20260925 02:43:32.567492 21490 raft_consensus.cc:2229] T 13843a3402bd4385a9a6cbaa5d11c96e P 15b827f1b5534fafa26e3142b9e9d192 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 02:43:32.567724 21492 raft_consensus.cc:2229] T 13843a3402bd4385a9a6cbaa5d11c96e P 323109284d62402a989a4cd4f8ec3c54 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 02:43:32.567827 21490 raft_consensus.cc:2258] T 13843a3402bd4385a9a6cbaa5d11c96e P 15b827f1b5534fafa26e3142b9e9d192 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:43:32.568104 21492 raft_consensus.cc:2258] T 13843a3402bd4385a9a6cbaa5d11c96e P 323109284d62402a989a4cd4f8ec3c54 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:43:32.568500 21491 tablet_replica.cc:330] T 13843a3402bd4385a9a6cbaa5d11c96e P 03b8c1c27f5b471498f53d8d326b61db: stopping tablet replica
I20260925 02:43:32.569679 21490 ts_tablet_manager.cc:1905] T 13843a3402bd4385a9a6cbaa5d11c96e P 15b827f1b5534fafa26e3142b9e9d192: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:43:32.569679 21492 ts_tablet_manager.cc:1905] T 13843a3402bd4385a9a6cbaa5d11c96e P 323109284d62402a989a4cd4f8ec3c54: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:43:32.570801 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 20140
I20260925 02:43:32.573092 21490 ts_tablet_manager.cc:1918] T 13843a3402bd4385a9a6cbaa5d11c96e P 15b827f1b5534fafa26e3142b9e9d192: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 02:43:32.573092 21492 ts_tablet_manager.cc:1918] T 13843a3402bd4385a9a6cbaa5d11c96e P 323109284d62402a989a4cd4f8ec3c54: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 02:43:32.573195 21490 log.cc:1196] T 13843a3402bd4385a9a6cbaa5d11c96e P 15b827f1b5534fafa26e3142b9e9d192: Deleting WAL directory at /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/ts-0/wal/wals/13843a3402bd4385a9a6cbaa5d11c96e
I20260925 02:43:32.573483 21490 ts_tablet_manager.cc:1939] T 13843a3402bd4385a9a6cbaa5d11c96e P 15b827f1b5534fafa26e3142b9e9d192: Deleting consensus metadata
I20260925 02:43:32.573616 21491 raft_consensus.cc:2229] T 13843a3402bd4385a9a6cbaa5d11c96e P 03b8c1c27f5b471498f53d8d326b61db [term 1 LEADER]: Raft consensus shutting down.
I20260925 02:43:32.574085 21491 raft_consensus.cc:2258] T 13843a3402bd4385a9a6cbaa5d11c96e P 03b8c1c27f5b471498f53d8d326b61db [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:43:32.574385 21492 log.cc:1196] T 13843a3402bd4385a9a6cbaa5d11c96e P 323109284d62402a989a4cd4f8ec3c54: Deleting WAL directory at /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/ts-2/wal/wals/13843a3402bd4385a9a6cbaa5d11c96e
I20260925 02:43:32.574761 21492 ts_tablet_manager.cc:1939] T 13843a3402bd4385a9a6cbaa5d11c96e P 323109284d62402a989a4cd4f8ec3c54: Deleting consensus metadata
I20260925 02:43:32.575629 21491 ts_tablet_manager.cc:1905] T 13843a3402bd4385a9a6cbaa5d11c96e P 03b8c1c27f5b471498f53d8d326b61db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:43:32.578117 21491 ts_tablet_manager.cc:1918] T 13843a3402bd4385a9a6cbaa5d11c96e P 03b8c1c27f5b471498f53d8d326b61db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 02:43:32.578212 21491 log.cc:1196] T 13843a3402bd4385a9a6cbaa5d11c96e P 03b8c1c27f5b471498f53d8d326b61db: Deleting WAL directory at /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/ts-1/wal/wals/13843a3402bd4385a9a6cbaa5d11c96e
I20260925 02:43:32.578505 21491 ts_tablet_manager.cc:1939] T 13843a3402bd4385a9a6cbaa5d11c96e P 03b8c1c27f5b471498f53d8d326b61db: Deleting consensus metadata
W20260925 02:43:32.578616 21042 connection.cc:535] server connection from 127.0.0.1:51240 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260925 02:43:32.579151 21323 connection.cc:535] server connection from 127.0.0.1:53490 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260925 02:43:32.590157 21448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:32.569608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56772 (local address 127.13.157.189:33117)
0925 02:43:32.569837 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:32.569841 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:32.579812 (+ 9971us) server_negotiation.cc:407] Connection header received
0925 02:43:32.579961 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:32.579965 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:32.580023 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:32.580106 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:32.580989 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:32.581661 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:32.583012 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:32.583423 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:32.583522 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:32.585491 (+ 1969us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:32.585496 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:32.585499 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:43:32.585528 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:32.587784 (+ 2256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:32.588583 (+ 799us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:32.588588 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:32.588590 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:32.588686 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:32.589070 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:32.589074 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:32.589076 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:32.589314 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:32.589421 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:32.589816 (+ 395us) server_negotiation.cc:299] Negotiation successful
0925 02:43:32.589959 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153}
I20260925 02:43:32.591570 20910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:32.582867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56776 (local address 127.13.157.189:33117)
0925 02:43:32.582992 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:32.582995 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:32.583372 (+ 377us) server_negotiation.cc:407] Connection header received
0925 02:43:32.583535 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:32.583538 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:32.583587 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:32.583637 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:32.584783 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:32.585542 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:32.586558 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:32.586848 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:32.586940 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:32.588434 (+ 1494us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:32.588438 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:32.588440 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:43:32.588466 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:32.589906 (+ 1440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:32.590326 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:32.590330 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:32.590332 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:32.590404 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:32.590690 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:32.590693 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:32.590695 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:32.590878 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:32.591059 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:32.591339 (+ 280us) server_negotiation.cc:299] Negotiation successful
0925 02:43:32.591427 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260925 02:43:32.591578 21161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:32.582699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34870 (local address 127.13.157.188:46853)
0925 02:43:32.582826 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:32.582830 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:32.582844 (+ 14us) server_negotiation.cc:407] Connection header received
0925 02:43:32.582884 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:32.582887 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:32.582959 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:32.583034 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:32.583998 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:32.584739 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:32.585811 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:32.586145 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:32.586214 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:32.587747 (+ 1533us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:32.587751 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:32.587754 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:43:32.587781 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:32.589477 (+ 1696us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:32.589934 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:32.589937 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:32.589939 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:32.590021 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:32.590592 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:32.590595 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:32.590597 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:32.590800 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:32.590897 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:32.591315 (+ 418us) server_negotiation.cc:299] Negotiation successful
0925 02:43:32.591427 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
W20260925 02:43:32.591759 21182 connection.cc:441] server connection from 127.0.0.1:60118 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:60118 (request call id 1) could send its response
I20260925 02:43:32.599682 21302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:32.569719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34864 (local address 127.13.157.188:46853)
0925 02:43:32.569920 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:32.569924 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:32.591733 (+ 21809us) server_negotiation.cc:407] Connection header received
0925 02:43:32.591944 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:32.591948 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:32.592000 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:32.592056 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:32.593035 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:32.593559 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:32.594636 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:32.594862 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:32.594991 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:32.596427 (+ 1436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:32.596430 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:32.596432 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:43:32.596452 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:32.597675 (+ 1223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:32.598141 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:32.598145 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:32.598147 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:32.598219 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:32.598531 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:32.598535 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:32.598537 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:32.598769 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:32.598873 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:32.599385 (+ 512us) server_negotiation.cc:299] Negotiation successful
0925 02:43:32.599515 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138}
I20260925 02:43:33.555042 21501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:33.553971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:34721 (local address 127.13.157.129:49407)
0925 02:43:33.554810 (+ 839us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:33.554900 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:34721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":771,"thread_start_us":81,"threads_started":1}
I20260925 02:43:33.555140 21500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:33.553913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:34721 (local address 127.13.157.131:54029)
0925 02:43:33.554334 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:33.555051 (+ 717us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:34721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":129,"threads_started":1}
W20260925 02:43:33.555388 21157 heartbeater.cc:644] Failed to heartbeat to 127.13.157.190:34721 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.13.157.190:34721: connect: Connection refused (error 111)
W20260925 02:43:33.555586 21436 heartbeater.cc:644] Failed to heartbeat to 127.13.157.190:34721 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.13.157.190:34721: connect: Connection refused (error 111)
I20260925 02:43:33.555608 21502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:33.554818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:34721 (local address 127.13.157.130:34161)
0925 02:43:33.555433 (+ 615us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:33.555517 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:34721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":545,"thread_start_us":84,"threads_started":1}
W20260925 02:43:33.555897 21296 heartbeater.cc:644] Failed to heartbeat to 127.13.157.190:34721 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.13.157.190:34721: connect: Connection refused (error 111)
I20260925 02:43:34.066715 21503 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1d39673cf6e84fd9a7012f26f96107e0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2b4937775ce34bed9ca35afdca0243b0)
I20260925 02:43:34.066885 21504 raft_consensus.cc:482] T 00000000000000000000000000000000 P 82186af4c8cb4282a83c2d2936d226ac [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2b4937775ce34bed9ca35afdca0243b0)
I20260925 02:43:34.066991 21503 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1d39673cf6e84fd9a7012f26f96107e0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "2b4937775ce34bed9ca35afdca0243b0" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 34721 } } peers { permanent_uuid: "1d39673cf6e84fd9a7012f26f96107e0" member_type: VOTER last_known_addr { host: "127.13.157.189" port: 33117 } attrs { promote: false } } peers { permanent_uuid: "82186af4c8cb4282a83c2d2936d226ac" member_type: VOTER last_known_addr { host: "127.13.157.188" port: 46853 } attrs { promote: false } }
I20260925 02:43:34.067090 21504 raft_consensus.cc:504] T 00000000000000000000000000000000 P 82186af4c8cb4282a83c2d2936d226ac [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "2b4937775ce34bed9ca35afdca0243b0" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 34721 } } peers { permanent_uuid: "1d39673cf6e84fd9a7012f26f96107e0" member_type: VOTER last_known_addr { host: "127.13.157.189" port: 33117 } attrs { promote: false } } peers { permanent_uuid: "82186af4c8cb4282a83c2d2936d226ac" member_type: VOTER last_known_addr { host: "127.13.157.188" port: 46853 } attrs { promote: false } }
I20260925 02:43:34.068002 21504 leader_election.cc:288] T 00000000000000000000000000000000 P 82186af4c8cb4282a83c2d2936d226ac [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2b4937775ce34bed9ca35afdca0243b0 (127.13.157.190:34721), 1d39673cf6e84fd9a7012f26f96107e0 (127.13.157.189:33117)
I20260925 02:43:34.067983 21503 leader_election.cc:288] T 00000000000000000000000000000000 P 1d39673cf6e84fd9a7012f26f96107e0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2b4937775ce34bed9ca35afdca0243b0 (127.13.157.190:34721), 82186af4c8cb4282a83c2d2936d226ac (127.13.157.188:46853)
I20260925 02:43:34.068692 21505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:34.068189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:34721 (local address 127.0.0.1:33444)
0925 02:43:34.068510 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:34.068588 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:34721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":78,"threads_started":1}
W20260925 02:43:34.068843 20957 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.157.190:34721: connect: Connection refused (error 111)
I20260925 02:43:34.068820 21506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:34.068191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:34721 (local address 127.0.0.1:33428)
0925 02:43:34.068633 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:34.068708 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:34721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":84,"threads_started":1}
W20260925 02:43:34.069059 20770 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.157.190:34721: connect: Connection refused (error 111)
I20260925 02:43:34.070024 21506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:34.069764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:34721 (local address 127.0.0.1:33458)
0925 02:43:34.069870 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:34.069943 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:34721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260925 02:43:34.070204 20770 leader_election.cc:334] T 00000000000000000000000000000000 P 1d39673cf6e84fd9a7012f26f96107e0 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 2b4937775ce34bed9ca35afdca0243b0 (127.13.157.190:34721): Network error: Client connection negotiation failed: client connection to 127.13.157.190:34721: connect: Connection refused (error 111)
I20260925 02:43:34.070329 21507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:34.069562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:34721 (local address 127.0.0.1:33456)
0925 02:43:34.070210 (+ 648us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:34.070264 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:34721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":583}
W20260925 02:43:34.070438 20957 leader_election.cc:334] T 00000000000000000000000000000000 P 82186af4c8cb4282a83c2d2936d226ac [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 2b4937775ce34bed9ca35afdca0243b0 (127.13.157.190:34721): Network error: Client connection negotiation failed: client connection to 127.13.157.190:34721: connect: Connection refused (error 111)
I20260925 02:43:34.072701 21509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:34.068520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.188:46853 (local address 127.0.0.1:34874)
0925 02:43:34.068737 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:34.068749 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:34.068849 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:34.069410 (+ 561us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:34.069413 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:34.069434 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:43:34.069531 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:34.069538 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:34.070499 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:34.070502 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:34.071715 (+ 1213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:34.071725 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:34.072397 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:34.072400 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:34.072464 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:34.072484 (+ 20us) client_negotiation.cc:769] Sending connection context
0925 02:43:34.072542 (+ 58us) client_negotiation.cc:240] Negotiation successful
0925 02:43:34.072605 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"mutex_wait_us":85,"thread_start_us":62,"threads_started":1}
I20260925 02:43:34.072753 21510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:34.068693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34874 (local address 127.13.157.188:46853)
0925 02:43:34.069114 (+ 421us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:34.069118 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:34.069176 (+ 58us) server_negotiation.cc:407] Connection header received
0925 02:43:34.069230 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:34.069233 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:34.069283 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:34.069360 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:43:34.069644 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:34.070405 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:34.071845 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:34.072292 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:34.072406 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:34.072457 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:34.072564 (+ 107us) server_negotiation.cc:299] Negotiation successful
0925 02:43:34.072632 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":48,"threads_started":1}
I20260925 02:43:34.073184 20983 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1d39673cf6e84fd9a7012f26f96107e0" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "82186af4c8cb4282a83c2d2936d226ac" is_pre_election: true
I20260925 02:43:34.073407 21505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:34.068306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.189:33117 (local address 127.0.0.1:56780)
0925 02:43:34.069210 (+ 904us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:34.069222 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:34.069340 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:34.069906 (+ 566us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:34.069910 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:34.069929 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:43:34.070039 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:34.070048 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:34.071108 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:34.071112 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:34.072158 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:34.072168 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:34.073038 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:34.073042 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:34.073129 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:34.073151 (+ 22us) client_negotiation.cc:769] Sending connection context
0925 02:43:34.073228 (+ 77us) client_negotiation.cc:240] Negotiation successful
0925 02:43:34.073298 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":862,"thread_start_us":72,"threads_started":1}
I20260925 02:43:34.073431 21508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:34.068386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56780 (local address 127.13.157.189:33117)
0925 02:43:34.069157 (+ 771us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:34.069161 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:34.069287 (+ 126us) server_negotiation.cc:407] Connection header received
0925 02:43:34.069685 (+ 398us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:34.069689 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:34.069730 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:34.069780 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:43:34.070180 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:34.071016 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:34.072277 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:34.072847 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:34.072956 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:34.073010 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:34.073254 (+ 244us) server_negotiation.cc:299] Negotiation successful
0925 02:43:34.073318 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":710,"thread_start_us":66,"threads_started":1}
I20260925 02:43:34.073400 20983 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 82186af4c8cb4282a83c2d2936d226ac [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1d39673cf6e84fd9a7012f26f96107e0 in term 1.
I20260925 02:43:34.073752 20770 leader_election.cc:302] T 00000000000000000000000000000000 P 1d39673cf6e84fd9a7012f26f96107e0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1d39673cf6e84fd9a7012f26f96107e0, 82186af4c8cb4282a83c2d2936d226ac; no voters: 2b4937775ce34bed9ca35afdca0243b0
I20260925 02:43:34.073838 20795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "82186af4c8cb4282a83c2d2936d226ac" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "1d39673cf6e84fd9a7012f26f96107e0" is_pre_election: true
I20260925 02:43:34.073969 21503 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1d39673cf6e84fd9a7012f26f96107e0 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260925 02:43:34.074028 21503 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1d39673cf6e84fd9a7012f26f96107e0 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 2b4937775ce34bed9ca35afdca0243b0)
I20260925 02:43:34.074071 21503 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1d39673cf6e84fd9a7012f26f96107e0 [term 1 FOLLOWER]: Advancing to term 2
I20260925 02:43:34.075119 21503 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1d39673cf6e84fd9a7012f26f96107e0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "2b4937775ce34bed9ca35afdca0243b0" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 34721 } } peers { permanent_uuid: "1d39673cf6e84fd9a7012f26f96107e0" member_type: VOTER last_known_addr { host: "127.13.157.189" port: 33117 } attrs { promote: false } } peers { permanent_uuid: "82186af4c8cb4282a83c2d2936d226ac" member_type: VOTER last_known_addr { host: "127.13.157.188" port: 46853 } attrs { promote: false } }
I20260925 02:43:34.075364 20795 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 1d39673cf6e84fd9a7012f26f96107e0 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 82186af4c8cb4282a83c2d2936d226ac in current term 2: Already voted for candidate 1d39673cf6e84fd9a7012f26f96107e0 in this term.
I20260925 02:43:34.075484 21503 leader_election.cc:288] T 00000000000000000000000000000000 P 1d39673cf6e84fd9a7012f26f96107e0 [CANDIDATE]: Term 2 election: Requested vote from peers 2b4937775ce34bed9ca35afdca0243b0 (127.13.157.190:34721), 82186af4c8cb4282a83c2d2936d226ac (127.13.157.188:46853)
I20260925 02:43:34.075790 21509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:34.075593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:34721 (local address 127.0.0.1:33460)
0925 02:43:34.075681 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:34.075731 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:34721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260925 02:43:34.076004 20983 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1d39673cf6e84fd9a7012f26f96107e0" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "82186af4c8cb4282a83c2d2936d226ac"
I20260925 02:43:34.076125 20983 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 82186af4c8cb4282a83c2d2936d226ac [term 1 FOLLOWER]: Advancing to term 2
I20260925 02:43:34.076258 20960 leader_election.cc:302] T 00000000000000000000000000000000 P 82186af4c8cb4282a83c2d2936d226ac [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 82186af4c8cb4282a83c2d2936d226ac; no voters: 1d39673cf6e84fd9a7012f26f96107e0, 2b4937775ce34bed9ca35afdca0243b0
I20260925 02:43:34.076792 21509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:34.076617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:34721 (local address 127.0.0.1:33472)
0925 02:43:34.076701 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:34.076744 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:34721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260925 02:43:34.076885 20770 leader_election.cc:334] T 00000000000000000000000000000000 P 1d39673cf6e84fd9a7012f26f96107e0 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 2b4937775ce34bed9ca35afdca0243b0 (127.13.157.190:34721): Network error: Client connection negotiation failed: client connection to 127.13.157.190:34721: connect: Connection refused (error 111)
I20260925 02:43:34.077188 20983 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 82186af4c8cb4282a83c2d2936d226ac [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1d39673cf6e84fd9a7012f26f96107e0 in term 2.
I20260925 02:43:34.077287 21504 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 82186af4c8cb4282a83c2d2936d226ac [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 02:43:34.077472 20770 leader_election.cc:302] T 00000000000000000000000000000000 P 1d39673cf6e84fd9a7012f26f96107e0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1d39673cf6e84fd9a7012f26f96107e0, 82186af4c8cb4282a83c2d2936d226ac; no voters: 2b4937775ce34bed9ca35afdca0243b0
I20260925 02:43:34.077674 21503 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1d39673cf6e84fd9a7012f26f96107e0 [term 2 FOLLOWER]: Leader election won for term 2
I20260925 02:43:34.077878 21503 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1d39673cf6e84fd9a7012f26f96107e0 [term 2 LEADER]: Becoming Leader. State: Replica: 1d39673cf6e84fd9a7012f26f96107e0, State: Running, Role: LEADER
I20260925 02:43:34.078128 21503 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1d39673cf6e84fd9a7012f26f96107e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "2b4937775ce34bed9ca35afdca0243b0" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 34721 } } peers { permanent_uuid: "1d39673cf6e84fd9a7012f26f96107e0" member_type: VOTER last_known_addr { host: "127.13.157.189" port: 33117 } attrs { promote: false } } peers { permanent_uuid: "82186af4c8cb4282a83c2d2936d226ac" member_type: VOTER last_known_addr { host: "127.13.157.188" port: 46853 } attrs { promote: false } }
I20260925 02:43:34.079403 21513 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d39673cf6e84fd9a7012f26f96107e0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1d39673cf6e84fd9a7012f26f96107e0. Latest consensus state: current_term: 2 leader_uuid: "1d39673cf6e84fd9a7012f26f96107e0" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "2b4937775ce34bed9ca35afdca0243b0" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 34721 } } peers { permanent_uuid: "1d39673cf6e84fd9a7012f26f96107e0" member_type: VOTER last_known_addr { host: "127.13.157.189" port: 33117 } attrs { promote: false } } peers { permanent_uuid: "82186af4c8cb4282a83c2d2936d226ac" member_type: VOTER last_known_addr { host: "127.13.157.188" port: 46853 } attrs { promote: false } } }
I20260925 02:43:34.079542 21513 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d39673cf6e84fd9a7012f26f96107e0 [sys.catalog]: This master's current role is: LEADER
I20260925 02:43:34.079912 21515 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:43:34.082187 21515 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:43:34.082531 21515 catalog_manager.cc:1259] Loaded cluster ID: 19dd28aa2b484a96bcdd464cb2d07319
I20260925 02:43:34.082572 21515 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 02:43:34.083153 21515 catalog_manager.cc:1504] Loading token signing keys...
I20260925 02:43:34.083494 21515 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 1d39673cf6e84fd9a7012f26f96107e0: Loaded TSK: 0
I20260925 02:43:34.084116 21515 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 02:43:34.110967 20787 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56776:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 02:43:34.113971 20787 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 02:43:34.117023 21509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:34.116794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:34721 (local address 127.0.0.1:33494)
0925 02:43:34.116907 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:34.116955 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:34721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260925 02:43:34.117071 20983 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 82186af4c8cb4282a83c2d2936d226ac [term 2 FOLLOWER]: Refusing update from remote peer 1d39673cf6e84fd9a7012f26f96107e0: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260925 02:43:34.117729 21509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:34.117529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:34721 (local address 127.0.0.1:33504)
0925 02:43:34.117628 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:34.117672 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:34721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260925 02:43:34.117816 21513 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1d39673cf6e84fd9a7012f26f96107e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "82186af4c8cb4282a83c2d2936d226ac" member_type: VOTER last_known_addr { host: "127.13.157.188" port: 46853 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260925 02:43:34.118010 20770 consensus_peers.cc:487] T 00000000000000000000000000000000 P 1d39673cf6e84fd9a7012f26f96107e0 -> Peer 2b4937775ce34bed9ca35afdca0243b0 (127.13.157.190:34721): Couldn't send request to peer 2b4937775ce34bed9ca35afdca0243b0. Status: Network error: Client connection negotiation failed: client connection to 127.13.157.190:34721: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260925 02:43:34.119525 21504 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82186af4c8cb4282a83c2d2936d226ac [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1d39673cf6e84fd9a7012f26f96107e0. Latest consensus state: current_term: 2 leader_uuid: "1d39673cf6e84fd9a7012f26f96107e0" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "2b4937775ce34bed9ca35afdca0243b0" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 34721 } } peers { permanent_uuid: "1d39673cf6e84fd9a7012f26f96107e0" member_type: VOTER last_known_addr { host: "127.13.157.189" port: 33117 } attrs { promote: false } } peers { permanent_uuid: "82186af4c8cb4282a83c2d2936d226ac" member_type: VOTER last_known_addr { host: "127.13.157.188" port: 46853 } attrs { promote: false } } }
I20260925 02:43:34.119665 21504 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82186af4c8cb4282a83c2d2936d226ac [sys.catalog]: This master's current role is: FOLLOWER
I20260925 02:43:34.120908 21513 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d39673cf6e84fd9a7012f26f96107e0 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "1d39673cf6e84fd9a7012f26f96107e0" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "2b4937775ce34bed9ca35afdca0243b0" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 34721 } } peers { permanent_uuid: "1d39673cf6e84fd9a7012f26f96107e0" member_type: VOTER last_known_addr { host: "127.13.157.189" port: 33117 } attrs { promote: false } } peers { permanent_uuid: "82186af4c8cb4282a83c2d2936d226ac" member_type: VOTER last_known_addr { host: "127.13.157.188" port: 46853 } attrs { promote: false } } }
I20260925 02:43:34.121073 21513 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d39673cf6e84fd9a7012f26f96107e0 [sys.catalog]: This master's current role is: LEADER
I20260925 02:43:34.121599 21504 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82186af4c8cb4282a83c2d2936d226ac [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "1d39673cf6e84fd9a7012f26f96107e0" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "2b4937775ce34bed9ca35afdca0243b0" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 34721 } } peers { permanent_uuid: "1d39673cf6e84fd9a7012f26f96107e0" member_type: VOTER last_known_addr { host: "127.13.157.189" port: 33117 } attrs { promote: false } } peers { permanent_uuid: "82186af4c8cb4282a83c2d2936d226ac" member_type: VOTER last_known_addr { host: "127.13.157.188" port: 46853 } attrs { promote: false } } }
I20260925 02:43:34.121733 21504 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82186af4c8cb4282a83c2d2936d226ac [sys.catalog]: This master's current role is: FOLLOWER
I20260925 02:43:34.131263 21523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:34.127229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53492 (local address 127.13.157.131:33075)
0925 02:43:34.127490 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:34.127495 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:34.127870 (+ 375us) server_negotiation.cc:407] Connection header received
0925 02:43:34.127919 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:34.127921 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:34.127960 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:34.128033 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:43:34.128252 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:34.128842 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:34.130210 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:34.130742 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:34.130851 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:34.130928 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:34.131075 (+ 147us) server_negotiation.cc:299] Negotiation successful
0925 02:43:34.131148 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":81,"threads_started":1}
I20260925 02:43:34.131255 21506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:34.127202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.131:33075 (local address 127.0.0.1:53492)
0925 02:43:34.127709 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:34.127722 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:34.127801 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:34.128057 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:34.128060 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:34.128076 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:43:34.128157 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:34.128162 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:34.128938 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:34.128942 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:34.130094 (+ 1152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:34.130104 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:34.130851 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:34.130854 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:34.130965 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:34.130988 (+ 23us) client_negotiation.cc:769] Sending connection context
0925 02:43:34.131051 (+ 63us) client_negotiation.cc:240] Negotiation successful
0925 02:43:34.131124 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":439}
I20260925 02:43:34.131412 21509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:34.126651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.129:46073 (local address 127.0.0.1:51242)
0925 02:43:34.126767 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:34.126781 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:34.126857 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:34.127701 (+ 844us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:34.127704 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:34.127723 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:43:34.127828 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:34.127835 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:34.128889 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:34.128892 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:34.129758 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:34.129767 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:34.131006 (+ 1239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:34.131009 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:34.131099 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:34.131125 (+ 26us) client_negotiation.cc:769] Sending connection context
0925 02:43:34.131169 (+ 44us) client_negotiation.cc:240] Negotiation successful
0925 02:43:34.131253 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42,"mutex_wait_us":21}
I20260925 02:43:34.131940 21522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:34.126942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51242 (local address 127.13.157.129:46073)
0925 02:43:34.127342 (+ 400us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:34.127347 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:34.127460 (+ 113us) server_negotiation.cc:407] Connection header received
0925 02:43:34.127521 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:34.127524 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:34.127573 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:34.127941 (+ 368us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:43:34.128056 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:34.128779 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:34.130252 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:34.130849 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:34.131665 (+ 816us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:34.131714 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:34.131747 (+ 33us) server_negotiation.cc:299] Negotiation successful
0925 02:43:34.131819 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":66,"threads_started":1}
I20260925 02:43:34.131994 21368 tablet_service.cc:1467] Processing CreateTablet for tablet 770103c0819f47c28c07bfa958672f72 (DEFAULT_TABLE table=test-table [id=b5195f3a6cbd4f1fbd4af94cad1580fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:43:34.132262 21368 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 770103c0819f47c28c07bfa958672f72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:43:34.132480 21090 tablet_service.cc:1467] Processing CreateTablet for tablet 770103c0819f47c28c07bfa958672f72 (DEFAULT_TABLE table=test-table [id=b5195f3a6cbd4f1fbd4af94cad1580fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:43:34.132737 21090 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 770103c0819f47c28c07bfa958672f72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:43:34.134927 21526 tablet_bootstrap.cc:492] T 770103c0819f47c28c07bfa958672f72 P 323109284d62402a989a4cd4f8ec3c54: Bootstrap starting.
I20260925 02:43:34.135188 21524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:34.127416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.130:36015 (local address 127.0.0.1:60124)
0925 02:43:34.127778 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:34.127790 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:34.127862 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:34.128340 (+ 478us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:34.128343 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:34.128360 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:43:34.128484 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:34.128489 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:34.129496 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:34.129499 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:34.130584 (+ 1085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:34.130593 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:34.131621 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:34.131625 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:34.131720 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:34.131740 (+ 20us) client_negotiation.cc:769] Sending connection context
0925 02:43:34.134973 (+ 3233us) client_negotiation.cc:240] Negotiation successful
0925 02:43:34.135058 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"mutex_wait_us":1,"thread_start_us":77,"threads_started":1}
I20260925 02:43:34.135524 21525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:34.127520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60124 (local address 127.13.157.130:36015)
0925 02:43:34.128051 (+ 531us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:34.128055 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:34.128067 (+ 12us) server_negotiation.cc:407] Connection header received
0925 02:43:34.128111 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:34.128114 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:34.128156 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:34.128232 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:43:34.128607 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:34.129353 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:34.130721 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:34.131328 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:34.131514 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:34.131566 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:34.135344 (+ 3778us) server_negotiation.cc:299] Negotiation successful
0925 02:43:34.135424 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":459,"thread_start_us":77,"threads_started":1}
I20260925 02:43:34.135782 21527 tablet_bootstrap.cc:492] T 770103c0819f47c28c07bfa958672f72 P 15b827f1b5534fafa26e3142b9e9d192: Bootstrap starting.
I20260925 02:43:34.135993 21228 tablet_service.cc:1467] Processing CreateTablet for tablet 770103c0819f47c28c07bfa958672f72 (DEFAULT_TABLE table=test-table [id=b5195f3a6cbd4f1fbd4af94cad1580fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:43:34.136075 21526 tablet_bootstrap.cc:654] T 770103c0819f47c28c07bfa958672f72 P 323109284d62402a989a4cd4f8ec3c54: Neither blocks nor log segments found. Creating new log.
I20260925 02:43:34.136260 21228 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 770103c0819f47c28c07bfa958672f72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:43:34.136590 21527 tablet_bootstrap.cc:654] T 770103c0819f47c28c07bfa958672f72 P 15b827f1b5534fafa26e3142b9e9d192: Neither blocks nor log segments found. Creating new log.
I20260925 02:43:34.137451 21526 tablet_bootstrap.cc:492] T 770103c0819f47c28c07bfa958672f72 P 323109284d62402a989a4cd4f8ec3c54: No bootstrap required, opened a new log
I20260925 02:43:34.137539 21526 ts_tablet_manager.cc:1397] T 770103c0819f47c28c07bfa958672f72 P 323109284d62402a989a4cd4f8ec3c54: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260925 02:43:34.137800 21527 tablet_bootstrap.cc:492] T 770103c0819f47c28c07bfa958672f72 P 15b827f1b5534fafa26e3142b9e9d192: No bootstrap required, opened a new log
I20260925 02:43:34.137883 21527 ts_tablet_manager.cc:1397] T 770103c0819f47c28c07bfa958672f72 P 15b827f1b5534fafa26e3142b9e9d192: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 02:43:34.137908 21526 raft_consensus.cc:348] T 770103c0819f47c28c07bfa958672f72 P 323109284d62402a989a4cd4f8ec3c54 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15b827f1b5534fafa26e3142b9e9d192" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 46073 } } peers { permanent_uuid: "323109284d62402a989a4cd4f8ec3c54" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 33075 } } peers { permanent_uuid: "03b8c1c27f5b471498f53d8d326b61db" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 36015 } }
I20260925 02:43:34.138034 21526 raft_consensus.cc:374] T 770103c0819f47c28c07bfa958672f72 P 323109284d62402a989a4cd4f8ec3c54 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:43:34.138069 21526 raft_consensus.cc:729] T 770103c0819f47c28c07bfa958672f72 P 323109284d62402a989a4cd4f8ec3c54 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 323109284d62402a989a4cd4f8ec3c54, State: Initialized, Role: FOLLOWER
I20260925 02:43:34.138196 21526 consensus_queue.cc:260] T 770103c0819f47c28c07bfa958672f72 P 323109284d62402a989a4cd4f8ec3c54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15b827f1b5534fafa26e3142b9e9d192" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 46073 } } peers { permanent_uuid: "323109284d62402a989a4cd4f8ec3c54" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 33075 } } peers { permanent_uuid: "03b8c1c27f5b471498f53d8d326b61db" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 36015 } }
I20260925 02:43:34.138304 21527 raft_consensus.cc:348] T 770103c0819f47c28c07bfa958672f72 P 15b827f1b5534fafa26e3142b9e9d192 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15b827f1b5534fafa26e3142b9e9d192" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 46073 } } peers { permanent_uuid: "323109284d62402a989a4cd4f8ec3c54" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 33075 } } peers { permanent_uuid: "03b8c1c27f5b471498f53d8d326b61db" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 36015 } }
I20260925 02:43:34.138419 21527 raft_consensus.cc:374] T 770103c0819f47c28c07bfa958672f72 P 15b827f1b5534fafa26e3142b9e9d192 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:43:34.138456 21527 raft_consensus.cc:729] T 770103c0819f47c28c07bfa958672f72 P 15b827f1b5534fafa26e3142b9e9d192 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15b827f1b5534fafa26e3142b9e9d192, State: Initialized, Role: FOLLOWER
I20260925 02:43:34.138513 21526 ts_tablet_manager.cc:1428] T 770103c0819f47c28c07bfa958672f72 P 323109284d62402a989a4cd4f8ec3c54: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260925 02:43:34.138599 21527 consensus_queue.cc:260] T 770103c0819f47c28c07bfa958672f72 P 15b827f1b5534fafa26e3142b9e9d192 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15b827f1b5534fafa26e3142b9e9d192" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 46073 } } peers { permanent_uuid: "323109284d62402a989a4cd4f8ec3c54" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 33075 } } peers { permanent_uuid: "03b8c1c27f5b471498f53d8d326b61db" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 36015 } }
I20260925 02:43:34.138957 21527 ts_tablet_manager.cc:1428] T 770103c0819f47c28c07bfa958672f72 P 15b827f1b5534fafa26e3142b9e9d192: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260925 02:43:34.139667 21533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:34.138971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:34721 (local address 127.13.157.131:34459)
0925 02:43:34.139504 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:34.139564 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:34721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":90,"threads_started":1}
I20260925 02:43:34.141084 21534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:34.140731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:34721 (local address 127.13.157.129:57851)
0925 02:43:34.140966 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:34.141015 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:34721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":105,"threads_started":1}
I20260925 02:43:34.141537 21437 heartbeater.cc:503] Master 127.13.157.189:33117 was elected leader, sending a full tablet report...
I20260925 02:43:34.141894 21158 heartbeater.cc:503] Master 127.13.157.189:33117 was elected leader, sending a full tablet report...
I20260925 02:43:34.141992 21531 tablet_bootstrap.cc:492] T 770103c0819f47c28c07bfa958672f72 P 03b8c1c27f5b471498f53d8d326b61db: Bootstrap starting.
I20260925 02:43:34.142138 21533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:34.141926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:34721 (local address 127.13.157.131:47041)
0925 02:43:34.142024 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:34.142078 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:34721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260925 02:43:34.143275 21534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:34.143077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:34721 (local address 127.13.157.129:42041)
0925 02:43:34.143161 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:34.143213 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:34721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260925 02:43:34.143659 21531 tablet_bootstrap.cc:654] T 770103c0819f47c28c07bfa958672f72 P 03b8c1c27f5b471498f53d8d326b61db: Neither blocks nor log segments found. Creating new log.
I20260925 02:43:34.144693 21531 tablet_bootstrap.cc:492] T 770103c0819f47c28c07bfa958672f72 P 03b8c1c27f5b471498f53d8d326b61db: No bootstrap required, opened a new log
I20260925 02:43:34.144784 21531 ts_tablet_manager.cc:1397] T 770103c0819f47c28c07bfa958672f72 P 03b8c1c27f5b471498f53d8d326b61db: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260925 02:43:34.145154 21531 raft_consensus.cc:348] T 770103c0819f47c28c07bfa958672f72 P 03b8c1c27f5b471498f53d8d326b61db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15b827f1b5534fafa26e3142b9e9d192" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 46073 } } peers { permanent_uuid: "323109284d62402a989a4cd4f8ec3c54" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 33075 } } peers { permanent_uuid: "03b8c1c27f5b471498f53d8d326b61db" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 36015 } }
I20260925 02:43:34.145259 21531 raft_consensus.cc:374] T 770103c0819f47c28c07bfa958672f72 P 03b8c1c27f5b471498f53d8d326b61db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:43:34.145284 21531 raft_consensus.cc:729] T 770103c0819f47c28c07bfa958672f72 P 03b8c1c27f5b471498f53d8d326b61db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03b8c1c27f5b471498f53d8d326b61db, State: Initialized, Role: FOLLOWER
I20260925 02:43:34.145380 21531 consensus_queue.cc:260] T 770103c0819f47c28c07bfa958672f72 P 03b8c1c27f5b471498f53d8d326b61db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15b827f1b5534fafa26e3142b9e9d192" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 46073 } } peers { permanent_uuid: "323109284d62402a989a4cd4f8ec3c54" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 33075 } } peers { permanent_uuid: "03b8c1c27f5b471498f53d8d326b61db" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 36015 } }
I20260925 02:43:34.145628 21531 ts_tablet_manager.cc:1428] T 770103c0819f47c28c07bfa958672f72 P 03b8c1c27f5b471498f53d8d326b61db: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260925 02:43:34.146921 21297 heartbeater.cc:503] Master 127.13.157.189:33117 was elected leader, sending a full tablet report...
I20260925 02:43:34.147244 21537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:34.145928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:34721 (local address 127.13.157.130:42285)
0925 02:43:34.146742 (+ 814us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:34.146799 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:34721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":89,"threads_started":1}
I20260925 02:43:34.148267 21537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:34.148087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:34721 (local address 127.13.157.130:47161)
0925 02:43:34.148162 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:34.148207 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:34721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260925 02:43:34.156740 21438 tablet.cc:2369] T 770103c0819f47c28c07bfa958672f72 P 323109284d62402a989a4cd4f8ec3c54: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 02:43:34.207497 21298 tablet.cc:2369] T 770103c0819f47c28c07bfa958672f72 P 03b8c1c27f5b471498f53d8d326b61db: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 02:43:34.252784 21530 raft_consensus.cc:482] T 770103c0819f47c28c07bfa958672f72 P 323109284d62402a989a4cd4f8ec3c54 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 02:43:34.252935 21530 raft_consensus.cc:504] T 770103c0819f47c28c07bfa958672f72 P 323109284d62402a989a4cd4f8ec3c54 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15b827f1b5534fafa26e3142b9e9d192" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 46073 } } peers { permanent_uuid: "323109284d62402a989a4cd4f8ec3c54" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 33075 } } peers { permanent_uuid: "03b8c1c27f5b471498f53d8d326b61db" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 36015 } }
I20260925 02:43:34.254101 21530 leader_election.cc:288] T 770103c0819f47c28c07bfa958672f72 P 323109284d62402a989a4cd4f8ec3c54 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 15b827f1b5534fafa26e3142b9e9d192 (127.13.157.129:46073), 03b8c1c27f5b471498f53d8d326b61db (127.13.157.130:36015)
I20260925 02:43:34.257920 21525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:34.254521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.131:52273 (local address 127.13.157.130:36015)
0925 02:43:34.254646 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:34.254650 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:34.254776 (+ 126us) server_negotiation.cc:407] Connection header received
0925 02:43:34.254881 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:34.254884 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:34.254950 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:34.255024 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:43:34.255266 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:34.255756 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:34.256932 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:34.257325 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:34.257431 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:34.257488 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:34.257719 (+ 231us) server_negotiation.cc:299] Negotiation successful
0925 02:43:34.257792 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260925 02:43:34.257921 21533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:34.254274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.129:46073 (local address 127.13.157.131:34585)
0925 02:43:34.254468 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:34.254483 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:34.254601 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:34.254877 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:34.254880 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:34.254926 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:43:34.255071 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:34.255078 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:34.255852 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:34.255855 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:34.256807 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:34.256816 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:34.257453 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:34.257456 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:34.257601 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:34.257624 (+ 23us) client_negotiation.cc:769] Sending connection context
0925 02:43:34.257687 (+ 63us) client_negotiation.cc:240] Negotiation successful
0925 02:43:34.257789 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"mutex_wait_us":93}
I20260925 02:43:34.257920 21522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:34.254369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.131:34585 (local address 127.13.157.129:46073)
0925 02:43:34.254513 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:34.254518 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:34.254537 (+ 19us) server_negotiation.cc:407] Connection header received
0925 02:43:34.254723 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:34.254727 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:34.254779 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:34.254852 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:43:34.255187 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:34.255737 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:34.256931 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:34.257322 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:34.257431 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:34.257487 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:34.257720 (+ 233us) server_negotiation.cc:299] Negotiation successful
0925 02:43:34.257796 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260925 02:43:34.257932 21538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:34.254329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.130:36015 (local address 127.13.157.131:52273)
0925 02:43:34.254685 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:34.254698 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:34.254795 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:34.255046 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:34.255048 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:34.255062 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:43:34.255161 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:34.255168 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:34.255863 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:34.255867 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:34.256808 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:34.256817 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:34.257458 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:34.257461 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:34.257599 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:34.257621 (+ 22us) client_negotiation.cc:769] Sending connection context
0925 02:43:34.257686 (+ 65us) client_negotiation.cc:240] Negotiation successful
0925 02:43:34.257779 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"mutex_wait_us":77,"thread_start_us":148,"threads_started":1}
I20260925 02:43:34.258380 21110 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "770103c0819f47c28c07bfa958672f72" candidate_uuid: "323109284d62402a989a4cd4f8ec3c54" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15b827f1b5534fafa26e3142b9e9d192" is_pre_election: true
I20260925 02:43:34.258533 21110 raft_consensus.cc:2454] T 770103c0819f47c28c07bfa958672f72 P 15b827f1b5534fafa26e3142b9e9d192 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 323109284d62402a989a4cd4f8ec3c54 in term 0.
I20260925 02:43:34.258985 21248 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "770103c0819f47c28c07bfa958672f72" candidate_uuid: "323109284d62402a989a4cd4f8ec3c54" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03b8c1c27f5b471498f53d8d326b61db" is_pre_election: true
I20260925 02:43:34.258987 21321 leader_election.cc:302] T 770103c0819f47c28c07bfa958672f72 P 323109284d62402a989a4cd4f8ec3c54 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15b827f1b5534fafa26e3142b9e9d192, 323109284d62402a989a4cd4f8ec3c54; no voters:
I20260925 02:43:34.259231 21248 raft_consensus.cc:2454] T 770103c0819f47c28c07bfa958672f72 P 03b8c1c27f5b471498f53d8d326b61db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 323109284d62402a989a4cd4f8ec3c54 in term 0.
I20260925 02:43:34.259315 21530 raft_consensus.cc:2781] T 770103c0819f47c28c07bfa958672f72 P 323109284d62402a989a4cd4f8ec3c54 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 02:43:34.259382 21530 raft_consensus.cc:482] T 770103c0819f47c28c07bfa958672f72 P 323109284d62402a989a4cd4f8ec3c54 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 02:43:34.259425 21530 raft_consensus.cc:3037] T 770103c0819f47c28c07bfa958672f72 P 323109284d62402a989a4cd4f8ec3c54 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:43:34.260499 21530 raft_consensus.cc:504] T 770103c0819f47c28c07bfa958672f72 P 323109284d62402a989a4cd4f8ec3c54 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15b827f1b5534fafa26e3142b9e9d192" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 46073 } } peers { permanent_uuid: "323109284d62402a989a4cd4f8ec3c54" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 33075 } } peers { permanent_uuid: "03b8c1c27f5b471498f53d8d326b61db" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 36015 } }
I20260925 02:43:34.260807 21530 leader_election.cc:288] T 770103c0819f47c28c07bfa958672f72 P 323109284d62402a989a4cd4f8ec3c54 [CANDIDATE]: Term 1 election: Requested vote from peers 15b827f1b5534fafa26e3142b9e9d192 (127.13.157.129:46073), 03b8c1c27f5b471498f53d8d326b61db (127.13.157.130:36015)
I20260925 02:43:34.261133 21248 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "770103c0819f47c28c07bfa958672f72" candidate_uuid: "323109284d62402a989a4cd4f8ec3c54" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03b8c1c27f5b471498f53d8d326b61db"
I20260925 02:43:34.261143 21110 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "770103c0819f47c28c07bfa958672f72" candidate_uuid: "323109284d62402a989a4cd4f8ec3c54" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15b827f1b5534fafa26e3142b9e9d192"
I20260925 02:43:34.261233 21248 raft_consensus.cc:3037] T 770103c0819f47c28c07bfa958672f72 P 03b8c1c27f5b471498f53d8d326b61db [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:43:34.261257 21110 raft_consensus.cc:3037] T 770103c0819f47c28c07bfa958672f72 P 15b827f1b5534fafa26e3142b9e9d192 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:43:34.262121 21248 raft_consensus.cc:2454] T 770103c0819f47c28c07bfa958672f72 P 03b8c1c27f5b471498f53d8d326b61db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 323109284d62402a989a4cd4f8ec3c54 in term 1.
I20260925 02:43:34.262130 21110 raft_consensus.cc:2454] T 770103c0819f47c28c07bfa958672f72 P 15b827f1b5534fafa26e3142b9e9d192 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 323109284d62402a989a4cd4f8ec3c54 in term 1.
I20260925 02:43:34.262373 21321 leader_election.cc:302] T 770103c0819f47c28c07bfa958672f72 P 323109284d62402a989a4cd4f8ec3c54 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 03b8c1c27f5b471498f53d8d326b61db, 15b827f1b5534fafa26e3142b9e9d192, 323109284d62402a989a4cd4f8ec3c54; no voters:
I20260925 02:43:34.262532 21530 raft_consensus.cc:2781] T 770103c0819f47c28c07bfa958672f72 P 323109284d62402a989a4cd4f8ec3c54 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:43:34.262739 21530 raft_consensus.cc:686] T 770103c0819f47c28c07bfa958672f72 P 323109284d62402a989a4cd4f8ec3c54 [term 1 LEADER]: Becoming Leader. State: Replica: 323109284d62402a989a4cd4f8ec3c54, State: Running, Role: LEADER
I20260925 02:43:34.263011 21530 consensus_queue.cc:237] T 770103c0819f47c28c07bfa958672f72 P 323109284d62402a989a4cd4f8ec3c54 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15b827f1b5534fafa26e3142b9e9d192" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 46073 } } peers { permanent_uuid: "323109284d62402a989a4cd4f8ec3c54" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 33075 } } peers { permanent_uuid: "03b8c1c27f5b471498f53d8d326b61db" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 36015 } }
I20260925 02:43:34.263404 21538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:34.263191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:34721 (local address 127.13.157.131:60855)
0925 02:43:34.263292 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:34.263342 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:34721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260925 02:43:34.265365 20787 catalog_manager.cc:5495] T 770103c0819f47c28c07bfa958672f72 P 323109284d62402a989a4cd4f8ec3c54 reported cstate change: term changed from 0 to 1, leader changed from <none> to 323109284d62402a989a4cd4f8ec3c54 (127.13.157.131). New cstate: current_term: 1 leader_uuid: "323109284d62402a989a4cd4f8ec3c54" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15b827f1b5534fafa26e3142b9e9d192" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 46073 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "323109284d62402a989a4cd4f8ec3c54" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "03b8c1c27f5b471498f53d8d326b61db" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 36015 } health_report { overall_health: UNKNOWN } } }
I20260925 02:43:34.286496 21523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:34.283098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53508 (local address 127.13.157.131:33075)
0925 02:43:34.283228 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:34.283233 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:34.283247 (+ 14us) server_negotiation.cc:407] Connection header received
0925 02:43:34.283393 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:34.283396 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:34.283452 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:34.283546 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 02:43:34.283785 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:34.284302 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:34.285134 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:34.285348 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:34.285440 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:34.285685 (+ 245us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 02:43:34.286128 (+ 443us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 02:43:34.286232 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:34.286302 (+ 70us) server_negotiation.cc:299] Negotiation successful
0925 02:43:34.286360 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260925 02:43:34.289193 21110 raft_consensus.cc:1261] T 770103c0819f47c28c07bfa958672f72 P 15b827f1b5534fafa26e3142b9e9d192 [term 1 FOLLOWER]: Refusing update from remote peer 323109284d62402a989a4cd4f8ec3c54: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:43:34.289438 21248 raft_consensus.cc:1261] T 770103c0819f47c28c07bfa958672f72 P 03b8c1c27f5b471498f53d8d326b61db [term 1 FOLLOWER]: Refusing update from remote peer 323109284d62402a989a4cd4f8ec3c54: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:43:34.289705 21539 consensus_queue.cc:1035] T 770103c0819f47c28c07bfa958672f72 P 323109284d62402a989a4cd4f8ec3c54 [LEADER]: Connected to new peer: Peer: permanent_uuid: "15b827f1b5534fafa26e3142b9e9d192" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 46073 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:43:34.289973 21530 consensus_queue.cc:1035] T 770103c0819f47c28c07bfa958672f72 P 323109284d62402a989a4cd4f8ec3c54 [LEADER]: Connected to new peer: Peer: permanent_uuid: "03b8c1c27f5b471498f53d8d326b61db" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 36015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:43:34.291608 21534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:34.291137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:34721 (local address 127.13.157.129:41825)
0925 02:43:34.291462 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:34.291525 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:34721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":258}
I20260925 02:43:34.292737 21537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:34.292312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:34721 (local address 127.13.157.130:60275)
0925 02:43:34.292608 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:34.292663 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:34721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":230}
I20260925 02:43:34.295401 21538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:34.295068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:34721 (local address 127.13.157.131:46275)
0925 02:43:34.295275 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:34.295330 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:34721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145}
I20260925 02:43:34.296787 21534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:34.296258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:34721 (local address 127.13.157.129:56549)
0925 02:43:34.296668 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:34.296720 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:34721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":341}
I20260925 02:43:34.297492 21537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:34.297252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:34721 (local address 127.13.157.130:45413)
0925 02:43:34.297347 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:34.297418 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:34721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260925 02:43:34.303241 21538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:34.303010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:34721 (local address 127.13.157.131:53781)
0925 02:43:34.303113 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:34.303165 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:34721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260925 02:43:34.303495 20785 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56776:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 02:43:34.303663 20785 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56776:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 02:43:34.307653 20785 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1d39673cf6e84fd9a7012f26f96107e0: Sending DeleteTablet for 3 replicas of tablet 770103c0819f47c28c07bfa958672f72
I20260925 02:43:34.308362 21368 tablet_service.cc:1514] Processing DeleteTablet for tablet 770103c0819f47c28c07bfa958672f72 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:43:34 UTC) from {username='oryx', principal='oryx/127.13.157.189@KRBTEST.COM'} at 127.0.0.1:53492
I20260925 02:43:34.308492 21228 tablet_service.cc:1514] Processing DeleteTablet for tablet 770103c0819f47c28c07bfa958672f72 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:43:34 UTC) from {username='oryx', principal='oryx/127.13.157.189@KRBTEST.COM'} at 127.0.0.1:60124
I20260925 02:43:34.308826 21090 tablet_service.cc:1514] Processing DeleteTablet for tablet 770103c0819f47c28c07bfa958672f72 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:43:34 UTC) from {username='oryx', principal='oryx/127.13.157.189@KRBTEST.COM'} at 127.0.0.1:51242
I20260925 02:43:34.308863 21551 tablet_replica.cc:330] T 770103c0819f47c28c07bfa958672f72 P 323109284d62402a989a4cd4f8ec3c54: stopping tablet replica
I20260925 02:43:34.309104 21551 raft_consensus.cc:2229] T 770103c0819f47c28c07bfa958672f72 P 323109284d62402a989a4cd4f8ec3c54 [term 1 LEADER]: Raft consensus shutting down.
I20260925 02:43:34.309381 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 20934
I20260925 02:43:34.309405 21551 raft_consensus.cc:2258] T 770103c0819f47c28c07bfa958672f72 P 323109284d62402a989a4cd4f8ec3c54 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:43:34.310726 21551 ts_tablet_manager.cc:1905] T 770103c0819f47c28c07bfa958672f72 P 323109284d62402a989a4cd4f8ec3c54: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:43:34.313345 21551 ts_tablet_manager.cc:1918] T 770103c0819f47c28c07bfa958672f72 P 323109284d62402a989a4cd4f8ec3c54: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 02:43:34.313421 21551 log.cc:1196] T 770103c0819f47c28c07bfa958672f72 P 323109284d62402a989a4cd4f8ec3c54: Deleting WAL directory at /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790304178325258-13942-0/minicluster-data/ts-2/wal/wals/770103c0819f47c28c07bfa958672f72
I20260925 02:43:34.313691 21551 ts_tablet_manager.cc:1939] T 770103c0819f47c28c07bfa958672f72 P 323109284d62402a989a4cd4f8ec3c54: Deleting consensus metadata
I20260925 02:43:34.314471 20772 catalog_manager.cc:4841] TS 323109284d62402a989a4cd4f8ec3c54 (127.13.157.131:33075): tablet 770103c0819f47c28c07bfa958672f72 (table test-table [id=b5195f3a6cbd4f1fbd4af94cad1580fe]) successfully deleted
I20260925 02:43:34.319165 21552 tablet_replica.cc:330] T 770103c0819f47c28c07bfa958672f72 P 03b8c1c27f5b471498f53d8d326b61db: stopping tablet replica
I20260925 02:43:34.319330 21552 raft_consensus.cc:2229] T 770103c0819f47c28c07bfa958672f72 P 03b8c1c27f5b471498f53d8d326b61db [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 02:43:34.319491 21552 raft_consensus.cc:2258] T 770103c0819f47c28c07bfa958672f72 P 03b8c1c27f5b471498f53d8d326b61db [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:43:34.319877 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 21164
I20260925 02:43:34.320081 21524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:34.319646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.129:46073 (local address 127.0.0.1:51254)
0925 02:43:34.319870 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:34.319929 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.129:46073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127}
W20260925 02:43:34.320662 20771 catalog_manager.cc:4584] TS 15b827f1b5534fafa26e3142b9e9d192 (127.13.157.129:46073): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 770103c0819f47c28c07bfa958672f72: Network error: Client connection negotiation failed: client connection to 127.13.157.129:46073: connect: Connection refused (error 111)
I20260925 02:43:34.326797 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 21303
I20260925 02:43:34.328167 21524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:34.327005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.130:36015 (local address 127.0.0.1:60130)
0925 02:43:34.327995 (+ 990us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:34.328067 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.130:36015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":897}
I20260925 02:43:34.332195 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 20645
I20260925 02:43:34.337504 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 20758
2026-09-25T02:43:34Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (5165 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790304178325258-13942-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 02:43:34 dist-test-slave-kznw krb5kdc[21556](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 02:43:34 dist-test-slave-kznw krb5kdc[21556](info): set up 2 sockets
Sep 25 02:43:34 dist-test-slave-kznw krb5kdc[21556](info): commencing operation
krb5kdc: starting...
W20260925 02:43:35.290944 13942 mini_kdc.cc:121] Time spent starting KDC: real 0.931s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790304178325258-13942-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790304178325258-13942-0/krb5kdc/test-user.keytab.
Sep 25 02:43:35 dist-test-slave-kznw krb5kdc[21556](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304215, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T02:43:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T02:43:35Z Disabled control of system clock
WARNING: no policy specified for oryx/127.13.157.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.157.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.157.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790304178325258-13942-0/krb5kdc/oryx_127.13.157.190.keytab.
Entry for principal oryx/127.13.157.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790304178325258-13942-0/krb5kdc/oryx_127.13.157.190.keytab.
I20260925 02:43:35.410315 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790304178325258-13942-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790304178325258-13942-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790304178325258-13942-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.157.190:43945
--webserver_interface=127.13.157.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790304178325258-13942-0/krb5kdc/oryx_127.13.157.190.keytab
--principal=oryx/127.13.157.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:38925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.157.190:43945
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:43:35.518225 21569 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:43:35.518494 21569 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:43:35.518550 21569 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:43:35.522599 21569 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:43:35.522668 21569 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260925 02:43:35.522693 21569 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:43:35.522716 21569 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:43:35.522739 21569 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:43:35.522768 21569 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:43:35.528512 21569 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:38925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790304178325258-13942-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790304178325258-13942-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.157.190:43945
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790304178325258-13942-0/krb5kdc/oryx_127.13.157.190.keytab
--principal=oryx/127.13.157.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.157.190:43945
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.157.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21569
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790304178325258-13942-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:43:35.529583 21569 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:43:35.530601 21569 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:43:35.538708 21575 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:35.538748 21577 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:35.538738 21574 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:43:35.538955 21569 server_base.cc:1034] running on GCE node
I20260925 02:43:35.539601 21569 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:43:35.540715 21569 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:43:35.541889 21569 hybrid_clock.cc:648] HybridClock initialized: now 1790304215541873 us; error 29 us; skew 500 ppm
Sep 25 02:43:35 dist-test-slave-kznw krb5kdc[21556](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304215, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.157.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:43:35.551810 21569 init.cc:377] Logged in from keytab as oryx/127.13.157.190@KRBTEST.COM (short username oryx)
I20260925 02:43:35.552847 21569 webserver.cc:460] Webserver started at http://127.13.157.190:42091/ using document root <none> and password file <none>
I20260925 02:43:35.553416 21569 fs_manager.cc:359] Metadata directory not provided
I20260925 02:43:35.553485 21569 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:43:35.553705 21569 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:43:35.555938 21569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790304178325258-13942-0/minicluster-data/master-0/data/instance:
uuid: "9120630de98a487eb499476860ce3215"
format_stamp: "Formatted at 2026-09-25 02:43:35 on dist-test-slave-kznw"
server_key: "8279203ebabf63b1b08c114cba3489cd"
server_key_iv: "ffb2bc09744b67ab8437d605724fa39a"
server_key_version: "encryptionkey@0"
I20260925 02:43:35.556504 21569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790304178325258-13942-0/minicluster-data/master-0/wal/instance:
uuid: "9120630de98a487eb499476860ce3215"
format_stamp: "Formatted at 2026-09-25 02:43:35 on dist-test-slave-kznw"
server_key: "8279203ebabf63b1b08c114cba3489cd"
server_key_iv: "ffb2bc09744b67ab8437d605724fa39a"
server_key_version: "encryptionkey@0"
I20260925 02:43:35.560395 21569 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260925 02:43:35.563066 21584 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:35.564380 21569 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 02:43:35.564505 21569 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790304178325258-13942-0/minicluster-data/master-0/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790304178325258-13942-0/minicluster-data/master-0/wal
uuid: "9120630de98a487eb499476860ce3215"
format_stamp: "Formatted at 2026-09-25 02:43:35 on dist-test-slave-kznw"
server_key: "8279203ebabf63b1b08c114cba3489cd"
server_key_iv: "ffb2bc09744b67ab8437d605724fa39a"
server_key_version: "encryptionkey@0"
I20260925 02:43:35.564615 21569 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790304178325258-13942-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790304178325258-13942-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790304178325258-13942-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:43:35.586122 21569 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:43:35.589208 21569 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:43:35.589382 21569 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:43:35.597201 21569 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.190:43945
I20260925 02:43:35.597223 21647 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.190:43945 every 8 connection(s)
I20260925 02:43:35.598129 21569 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
I20260925 02:43:35.600950 21648 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:43:35.606423 21648 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9120630de98a487eb499476860ce3215: Bootstrap starting.
I20260925 02:43:35.606544 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 21569
I20260925 02:43:35.606699 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790304178325258-13942-0/minicluster-data/master-0/wal/instance
I20260925 02:43:35.607050 13942 external_mini_cluster.cc:1428] Setting key a8530a149095499b9aa63b66901ea3e7
I20260925 02:43:35.609370 21648 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9120630de98a487eb499476860ce3215: Neither blocks nor log segments found. Creating new log.
I20260925 02:43:35.610186 21648 log.cc:824] T 00000000000000000000000000000000 P 9120630de98a487eb499476860ce3215: Log is configured to *not* fsync() on all Append() calls
I20260925 02:43:35.612331 21648 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9120630de98a487eb499476860ce3215: No bootstrap required, opened a new log
Sep 25 02:43:35 dist-test-slave-kznw krb5kdc[21556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304215, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM
I20260925 02:43:35.616748 21648 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9120630de98a487eb499476860ce3215 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9120630de98a487eb499476860ce3215" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 43945 } }
I20260925 02:43:35.617057 21648 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9120630de98a487eb499476860ce3215 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:43:35.617120 21648 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9120630de98a487eb499476860ce3215 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9120630de98a487eb499476860ce3215, State: Initialized, Role: FOLLOWER
I20260925 02:43:35.617571 21648 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9120630de98a487eb499476860ce3215 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9120630de98a487eb499476860ce3215" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 43945 } }
I20260925 02:43:35.617722 21648 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9120630de98a487eb499476860ce3215 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 02:43:35.617797 21648 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9120630de98a487eb499476860ce3215 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 02:43:35.617914 21648 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9120630de98a487eb499476860ce3215 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:43:35.618885 21648 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9120630de98a487eb499476860ce3215 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9120630de98a487eb499476860ce3215" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 43945 } }
I20260925 02:43:35.619295 21648 leader_election.cc:302] T 00000000000000000000000000000000 P 9120630de98a487eb499476860ce3215 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9120630de98a487eb499476860ce3215; no voters:
I20260925 02:43:35.619654 21648 leader_election.cc:288] T 00000000000000000000000000000000 P 9120630de98a487eb499476860ce3215 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 02:43:35.619782 21653 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9120630de98a487eb499476860ce3215 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:43:35.619987 21653 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9120630de98a487eb499476860ce3215 [term 1 LEADER]: Becoming Leader. State: Replica: 9120630de98a487eb499476860ce3215, State: Running, Role: LEADER
I20260925 02:43:35.620304 21653 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9120630de98a487eb499476860ce3215 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9120630de98a487eb499476860ce3215" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 43945 } }
I20260925 02:43:35.621116 21648 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9120630de98a487eb499476860ce3215 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:43:35.622037 21654 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9120630de98a487eb499476860ce3215 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9120630de98a487eb499476860ce3215" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9120630de98a487eb499476860ce3215" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 43945 } } }
I20260925 02:43:35.622159 21654 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9120630de98a487eb499476860ce3215 [sys.catalog]: This master's current role is: LEADER
I20260925 02:43:35.622020 21655 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9120630de98a487eb499476860ce3215 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9120630de98a487eb499476860ce3215. Latest consensus state: current_term: 1 leader_uuid: "9120630de98a487eb499476860ce3215" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9120630de98a487eb499476860ce3215" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 43945 } } }
I20260925 02:43:35.622260 21655 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9120630de98a487eb499476860ce3215 [sys.catalog]: This master's current role is: LEADER
I20260925 02:43:35.622485 21651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:35.608456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48200 (local address 127.13.157.190:43945)
0925 02:43:35.608908 (+ 452us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:35.608924 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:35.608960 (+ 36us) server_negotiation.cc:407] Connection header received
0925 02:43:35.609655 (+ 695us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:35.609675 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:35.610039 (+ 364us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:35.610357 (+ 318us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:35.611309 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:35.612316 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:35.613473 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:35.613941 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:35.614054 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:35.616927 (+ 2873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:35.616931 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:35.616944 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:43:35.616971 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:35.618682 (+ 1711us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:35.619192 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:35.619196 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:35.619201 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:35.619310 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:35.619628 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:35.619631 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:35.619633 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:35.620204 (+ 571us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:35.620401 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:35.621120 (+ 719us) server_negotiation.cc:299] Negotiation successful
0925 02:43:35.621442 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":310,"thread_start_us":133,"threads_started":1}
I20260925 02:43:35.622548 21662 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:43:35.626343 21662 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:43:35.632210 21662 catalog_manager.cc:1347] Generated new cluster ID: 8d35d2de1e0d4b1582ba3892f782d53d
I20260925 02:43:35.632283 21662 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 02:43:35.660535 21662 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 02:43:35.661888 21662 catalog_manager.cc:1504] Loading token signing keys...
I20260925 02:43:35.672116 21662 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9120630de98a487eb499476860ce3215: Generated new TSK 0
I20260925 02:43:35.672966 21662 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.13.157.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.157.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.157.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790304178325258-13942-0/krb5kdc/oryx_127.13.157.129.keytab.
Entry for principal oryx/127.13.157.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790304178325258-13942-0/krb5kdc/oryx_127.13.157.129.keytab.
I20260925 02:43:35.705552 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790304178325258-13942-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790304178325258-13942-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790304178325258-13942-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790304178325258-13942-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.157.129:0
--local_ip_for_outbound_sockets=127.13.157.129
--webserver_interface=127.13.157.129
--webserver_port=0
--tserver_master_addrs=127.13.157.190:43945
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790304178325258-13942-0/krb5kdc/oryx_127.13.157.129.keytab
--principal=oryx/127.13.157.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:38925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:43:35.816202 21674 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:43:35.816470 21674 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:43:35.816514 21674 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:43:35.820608 21674 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:43:35.820667 21674 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:43:35.820713 21674 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 02:43:35.820770 21674 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.157.129
I20260925 02:43:35.826241 21674 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:38925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790304178325258-13942-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790304178325258-13942-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790304178325258-13942-0/krb5kdc/oryx_127.13.157.129.keytab
--principal=oryx/127.13.157.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.157.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790304178325258-13942-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.157.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.157.190:43945
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21674
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.157.129
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790304178325258-13942-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:43:35.827389 21674 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:43:35.828495 21674 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:43:35.838227 21682 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:35.838292 21679 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:35.838241 21680 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:43:35.838521 21674 server_base.cc:1034] running on GCE node
I20260925 02:43:35.838986 21674 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:43:35.839666 21674 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:43:35.840847 21674 hybrid_clock.cc:648] HybridClock initialized: now 1790304215840801 us; error 59 us; skew 500 ppm
Sep 25 02:43:35 dist-test-slave-kznw krb5kdc[21556](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304215, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.157.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:43:35.843676 21674 init.cc:377] Logged in from keytab as oryx/127.13.157.129@KRBTEST.COM (short username oryx)
I20260925 02:43:35.844596 21674 webserver.cc:460] Webserver started at http://127.13.157.129:43335/ using document root <none> and password file <none>
I20260925 02:43:35.845131 21674 fs_manager.cc:359] Metadata directory not provided
I20260925 02:43:35.845193 21674 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:43:35.845374 21674 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:43:35.847553 21674 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790304178325258-13942-0/minicluster-data/ts-0/data/instance:
uuid: "77e5b44255714c0db6e1fa4e7130c90f"
format_stamp: "Formatted at 2026-09-25 02:43:35 on dist-test-slave-kznw"
server_key: "9e7f9a9146ebf67e9d54b2d11f8b970d"
server_key_iv: "80534f1a6ac527dcc8b9b1f21c3d5a5d"
server_key_version: "encryptionkey@0"
I20260925 02:43:35.848085 21674 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790304178325258-13942-0/minicluster-data/ts-0/wal/instance:
uuid: "77e5b44255714c0db6e1fa4e7130c90f"
format_stamp: "Formatted at 2026-09-25 02:43:35 on dist-test-slave-kznw"
server_key: "9e7f9a9146ebf67e9d54b2d11f8b970d"
server_key_iv: "80534f1a6ac527dcc8b9b1f21c3d5a5d"
server_key_version: "encryptionkey@0"
I20260925 02:43:35.852738 21674 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260925 02:43:35.855257 21689 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:35.856452 21674 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260925 02:43:35.856575 21674 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790304178325258-13942-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790304178325258-13942-0/minicluster-data/ts-0/wal
uuid: "77e5b44255714c0db6e1fa4e7130c90f"
format_stamp: "Formatted at 2026-09-25 02:43:35 on dist-test-slave-kznw"
server_key: "9e7f9a9146ebf67e9d54b2d11f8b970d"
server_key_iv: "80534f1a6ac527dcc8b9b1f21c3d5a5d"
server_key_version: "encryptionkey@0"
I20260925 02:43:35.856684 21674 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790304178325258-13942-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790304178325258-13942-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790304178325258-13942-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:43:35.873178 21674 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:43:35.876145 21674 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:43:35.876317 21674 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:43:35.877810 21674 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:43:35.877864 21674 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:35.877923 21674 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:43:35.877951 21674 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 25 02:43:35 dist-test-slave-kznw krb5kdc[21556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304215, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.157.129@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM
I20260925 02:43:35.887938 21674 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.129:40507
I20260925 02:43:35.887989 21806 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.129:40507 every 8 connection(s)
I20260925 02:43:35.888824 21674 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790304178325258-13942-0/minicluster-data/ts-0/data/info.pb
I20260925 02:43:35.891708 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 21674
I20260925 02:43:35.891845 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790304178325258-13942-0/minicluster-data/ts-0/wal/instance
I20260925 02:43:35.891924 21651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:35.878647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.129:38015 (local address 127.13.157.190:43945)
0925 02:43:35.878769 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:35.878773 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:35.879916 (+ 1143us) server_negotiation.cc:407] Connection header received
0925 02:43:35.881012 (+ 1096us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:35.881016 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:35.881089 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:35.881149 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:35.882844 (+ 1695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:35.883707 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:35.884974 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:35.885301 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:35.885396 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:35.888270 (+ 2874us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:35.888274 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:35.888278 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:43:35.888306 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:35.889810 (+ 1504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:35.890296 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:35.890299 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:35.890301 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:35.890395 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:35.890729 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:35.890732 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:35.890734 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:35.891012 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:35.891103 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:35.891632 (+ 529us) server_negotiation.cc:299] Negotiation successful
0925 02:43:35.891753 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260925 02:43:35.892182 13942 external_mini_cluster.cc:1428] Setting key b455b0bb6cc1dc54b77e98fb35a1bd27
I20260925 02:43:35.892879 21698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:35.879240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:43945 (local address 127.13.157.129:38015)
0925 02:43:35.879748 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:35.879795 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:35.880767 (+ 972us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:35.881575 (+ 808us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:35.881584 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:35.881989 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:35.882665 (+ 676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:35.882685 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:35.883834 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:35.883838 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:35.884853 (+ 1015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:35.884863 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:35.885462 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:35.885467 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:35.885638 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:35.886225 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:35.886246 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:35.888089 (+ 1843us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:43:35.889891 (+ 1802us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:35.889895 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:35.889905 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:35.890151 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:35.890483 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:35.890485 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:35.890487 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:35.890633 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:35.891124 (+ 491us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:43:35.891130 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:43:35.891360 (+ 230us) client_negotiation.cc:769] Sending connection context
0925 02:43:35.891567 (+ 207us) client_negotiation.cc:240] Negotiation successful
0925 02:43:35.891905 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":140,"threads_started":1}
I20260925 02:43:35.895418 21807 heartbeater.cc:347] Connected to a master server at 127.13.157.190:43945
I20260925 02:43:35.895571 21807 heartbeater.cc:464] Registering TS with master...
I20260925 02:43:35.896060 21807 heartbeater.cc:511] Master 127.13.157.190:43945 requested a full tablet report, sending...
I20260925 02:43:35.897490 21600 ts_manager.cc:194] Registered new tserver with Master: 77e5b44255714c0db6e1fa4e7130c90f (127.13.157.129:40507)
I20260925 02:43:35.898676 21600 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.157.129@KRBTEST.COM'} at 127.13.157.129:38015
WARNING: no policy specified for oryx/127.13.157.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.157.130@KRBTEST.COM" created.
I20260925 02:43:35.904718 21651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:35.896179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.129:57837 (local address 127.13.157.190:43945)
0925 02:43:35.896287 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:35.896291 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:35.898444 (+ 2153us) server_negotiation.cc:407] Connection header received
0925 02:43:35.898552 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:35.898555 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:35.898599 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:35.898680 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:35.899216 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:35.899842 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:35.900698 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:35.900919 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:35.901000 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:35.902025 (+ 1025us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:35.902029 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:35.902031 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:43:35.902056 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:35.903217 (+ 1161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:35.903586 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:35.903590 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:35.903592 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:35.903668 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:35.903937 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:35.903940 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:35.903942 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:35.904119 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:35.904204 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:35.904443 (+ 239us) server_negotiation.cc:299] Negotiation successful
0925 02:43:35.904562 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260925 02:43:35.904749 21814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:35.896109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:43945 (local address 127.13.157.129:57837)
0925 02:43:35.898345 (+ 2236us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:35.898359 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:35.898464 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:35.898710 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:35.898713 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:35.898960 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:35.899100 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:35.899108 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:35.899960 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:35.899963 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:35.900592 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:35.900601 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:35.901020 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:35.901024 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:35.901109 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:35.901546 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:35.901557 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:35.901861 (+ 304us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:43:35.903311 (+ 1450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:35.903315 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:35.903317 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:35.903498 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:35.903744 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:35.903746 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:35.903748 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:35.903826 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:35.904205 (+ 379us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:43:35.904208 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:43:35.904293 (+ 85us) client_negotiation.cc:769] Sending connection context
0925 02:43:35.904381 (+ 88us) client_negotiation.cc:240] Negotiation successful
0925 02:43:35.904564 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2145,"mutex_wait_us":40,"thread_start_us":95,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.157.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790304178325258-13942-0/krb5kdc/oryx_127.13.157.130.keytab.
Entry for principal oryx/127.13.157.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790304178325258-13942-0/krb5kdc/oryx_127.13.157.130.keytab.
I20260925 02:43:35.921999 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790304178325258-13942-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790304178325258-13942-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790304178325258-13942-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790304178325258-13942-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.157.130:0
--local_ip_for_outbound_sockets=127.13.157.130
--webserver_interface=127.13.157.130
--webserver_port=0
--tserver_master_addrs=127.13.157.190:43945
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790304178325258-13942-0/krb5kdc/oryx_127.13.157.130.keytab
--principal=oryx/127.13.157.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:38925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:43:36.032354 21816 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:43:36.032634 21816 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:43:36.032696 21816 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:43:36.036731 21816 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:43:36.036788 21816 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:43:36.036844 21816 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 02:43:36.036899 21816 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.157.130
I20260925 02:43:36.042158 21816 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:38925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790304178325258-13942-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790304178325258-13942-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790304178325258-13942-0/krb5kdc/oryx_127.13.157.130.keytab
--principal=oryx/127.13.157.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.157.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790304178325258-13942-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.157.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.157.190:43945
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21816
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.157.130
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790304178325258-13942-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:43:36.043347 21816 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:43:36.044442 21816 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 02:43:36.053900 21816 server_base.cc:1034] running on GCE node
W20260925 02:43:36.053854 21821 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:36.053922 21824 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:36.053841 21822 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:43:36.054544 21816 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:43:36.055177 21816 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:43:36.056331 21816 hybrid_clock.cc:648] HybridClock initialized: now 1790304216056314 us; error 26 us; skew 500 ppm
Sep 25 02:43:36 dist-test-slave-kznw krb5kdc[21556](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304216, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.157.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:43:36.058969 21816 init.cc:377] Logged in from keytab as oryx/127.13.157.130@KRBTEST.COM (short username oryx)
I20260925 02:43:36.059861 21816 webserver.cc:460] Webserver started at http://127.13.157.130:46735/ using document root <none> and password file <none>
I20260925 02:43:36.060396 21816 fs_manager.cc:359] Metadata directory not provided
I20260925 02:43:36.060470 21816 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:43:36.060673 21816 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:43:36.062743 21816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790304178325258-13942-0/minicluster-data/ts-1/data/instance:
uuid: "eb7b2024cae043fd8827ea003fbbcdbf"
format_stamp: "Formatted at 2026-09-25 02:43:36 on dist-test-slave-kznw"
server_key: "eaac971a478a1d4ab742d585f650e4fb"
server_key_iv: "53532b27ea7ce47fee2da4db6bff170c"
server_key_version: "encryptionkey@0"
I20260925 02:43:36.063354 21816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790304178325258-13942-0/minicluster-data/ts-1/wal/instance:
uuid: "eb7b2024cae043fd8827ea003fbbcdbf"
format_stamp: "Formatted at 2026-09-25 02:43:36 on dist-test-slave-kznw"
server_key: "eaac971a478a1d4ab742d585f650e4fb"
server_key_iv: "53532b27ea7ce47fee2da4db6bff170c"
server_key_version: "encryptionkey@0"
I20260925 02:43:36.066900 21816 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260925 02:43:36.069540 21831 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:36.070888 21816 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260925 02:43:36.071075 21816 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790304178325258-13942-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790304178325258-13942-0/minicluster-data/ts-1/wal
uuid: "eb7b2024cae043fd8827ea003fbbcdbf"
format_stamp: "Formatted at 2026-09-25 02:43:36 on dist-test-slave-kznw"
server_key: "eaac971a478a1d4ab742d585f650e4fb"
server_key_iv: "53532b27ea7ce47fee2da4db6bff170c"
server_key_version: "encryptionkey@0"
I20260925 02:43:36.071182 21816 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790304178325258-13942-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790304178325258-13942-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790304178325258-13942-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:43:36.083870 21816 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:43:36.086805 21816 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:43:36.087028 21816 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:43:36.088506 21816 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:43:36.088559 21816 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:36.088608 21816 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:43:36.088640 21816 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 25 02:43:36 dist-test-slave-kznw krb5kdc[21556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304216, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.157.130@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM
I20260925 02:43:36.099531 21816 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.130:33705
I20260925 02:43:36.099581 21947 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.130:33705 every 8 connection(s)
I20260925 02:43:36.100508 21816 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790304178325258-13942-0/minicluster-data/ts-1/data/info.pb
I20260925 02:43:36.102059 21651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:36.089230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.130:55259 (local address 127.13.157.190:43945)
0925 02:43:36.089395 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:36.089400 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:36.090168 (+ 768us) server_negotiation.cc:407] Connection header received
0925 02:43:36.091241 (+ 1073us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:36.091244 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:36.091305 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:36.091360 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:36.092715 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:36.093297 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:36.094603 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:36.094921 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:36.095021 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:36.098434 (+ 3413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:36.098438 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:36.098442 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:43:36.098470 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:36.100068 (+ 1598us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:36.100528 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:36.100532 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:36.100534 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:36.100624 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:36.100923 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:36.100927 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:36.100929 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:36.101163 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:36.101256 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:36.101771 (+ 515us) server_negotiation.cc:299] Negotiation successful
0925 02:43:36.101896 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260925 02:43:36.102800 21841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:36.089569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:43945 (local address 127.13.157.130:55259)
0925 02:43:36.090005 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:36.090055 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:36.090991 (+ 936us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:36.091741 (+ 750us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:36.091747 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:36.092091 (+ 344us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:36.092551 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:36.092566 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:36.093426 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:36.093428 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:36.094494 (+ 1066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:36.094504 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:36.095071 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:36.095075 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:36.095265 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:36.095860 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:36.095885 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:36.098272 (+ 2387us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:43:36.100173 (+ 1901us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:36.100180 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:36.100194 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:36.100433 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:36.100698 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:36.100702 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:36.100704 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:36.100838 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:36.101250 (+ 412us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:43:36.101257 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:43:36.101530 (+ 273us) client_negotiation.cc:769] Sending connection context
0925 02:43:36.101711 (+ 181us) client_negotiation.cc:240] Negotiation successful
0925 02:43:36.102015 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":107,"threads_started":1}
I20260925 02:43:36.104053 21948 heartbeater.cc:347] Connected to a master server at 127.13.157.190:43945
I20260925 02:43:36.104202 21948 heartbeater.cc:464] Registering TS with master...
I20260925 02:43:36.104696 21948 heartbeater.cc:511] Master 127.13.157.190:43945 requested a full tablet report, sending...
I20260925 02:43:36.105748 21601 ts_manager.cc:194] Registered new tserver with Master: eb7b2024cae043fd8827ea003fbbcdbf (127.13.157.130:33705)
I20260925 02:43:36.106614 21601 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.157.130@KRBTEST.COM'} at 127.13.157.130:55259
I20260925 02:43:36.108078 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 21816
I20260925 02:43:36.108227 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790304178325258-13942-0/minicluster-data/ts-1/wal/instance
I20260925 02:43:36.108608 13942 external_mini_cluster.cc:1428] Setting key c086bd306da037609d68ffafdc7aced1
I20260925 02:43:36.112807 21651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:36.106076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.130:49823 (local address 127.13.157.190:43945)
0925 02:43:36.106196 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:36.106201 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:36.106457 (+ 256us) server_negotiation.cc:407] Connection header received
0925 02:43:36.106590 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:36.106592 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:36.106631 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:36.106740 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:36.107210 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:36.107690 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:36.108591 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:36.108831 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:36.108917 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:36.109893 (+ 976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:36.109897 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:36.109900 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:43:36.109924 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:36.111321 (+ 1397us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:36.111696 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:36.111699 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:36.111701 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:36.111775 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:36.112011 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:36.112014 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:36.112016 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:36.112232 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:36.112316 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:36.112555 (+ 239us) server_negotiation.cc:299] Negotiation successful
0925 02:43:36.112658 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260925 02:43:36.112818 21954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:36.105988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:43945 (local address 127.13.157.130:49823)
0925 02:43:36.106363 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:36.106378 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:36.106498 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:36.106771 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:36.106773 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:36.107000 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:36.107104 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:36.107110 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:36.107818 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:36.107821 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:36.108489 (+ 668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:36.108498 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:36.108928 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:36.108931 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:36.109019 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:36.109450 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:36.109461 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:36.109788 (+ 327us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:43:36.111421 (+ 1633us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:36.111424 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:36.111426 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:36.111605 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:36.111850 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:36.111853 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:36.111855 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:36.111933 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:36.112318 (+ 385us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:43:36.112322 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:43:36.112410 (+ 88us) client_negotiation.cc:769] Sending connection context
0925 02:43:36.112492 (+ 82us) client_negotiation.cc:240] Negotiation successful
0925 02:43:36.112628 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":94,"threads_started":1}
WARNING: no policy specified for oryx/127.13.157.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.157.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.157.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790304178325258-13942-0/krb5kdc/oryx_127.13.157.131.keytab.
Entry for principal oryx/127.13.157.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790304178325258-13942-0/krb5kdc/oryx_127.13.157.131.keytab.
I20260925 02:43:36.134140 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790304178325258-13942-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790304178325258-13942-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790304178325258-13942-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790304178325258-13942-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.157.131:0
--local_ip_for_outbound_sockets=127.13.157.131
--webserver_interface=127.13.157.131
--webserver_port=0
--tserver_master_addrs=127.13.157.190:43945
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790304178325258-13942-0/krb5kdc/oryx_127.13.157.131.keytab
--principal=oryx/127.13.157.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:38925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:43:36.243575 21957 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:43:36.243832 21957 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:43:36.243875 21957 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:43:36.248040 21957 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:43:36.248109 21957 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:43:36.248164 21957 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 02:43:36.248216 21957 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.157.131
I20260925 02:43:36.253479 21957 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:38925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790304178325258-13942-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790304178325258-13942-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790304178325258-13942-0/krb5kdc/oryx_127.13.157.131.keytab
--principal=oryx/127.13.157.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.157.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790304178325258-13942-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.157.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.157.190:43945
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21957
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.157.131
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790304178325258-13942-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:43:36.254496 21957 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:43:36.255512 21957 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 02:43:36.265101 21957 server_base.cc:1034] running on GCE node
W20260925 02:43:36.265031 21963 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:36.265012 21962 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:36.265017 21965 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:43:36.265686 21957 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:43:36.266278 21957 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:43:36.267426 21957 hybrid_clock.cc:648] HybridClock initialized: now 1790304216267408 us; error 32 us; skew 500 ppm
Sep 25 02:43:36 dist-test-slave-kznw krb5kdc[21556](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304216, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.157.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:43:36.270054 21957 init.cc:377] Logged in from keytab as oryx/127.13.157.131@KRBTEST.COM (short username oryx)
I20260925 02:43:36.270893 21957 webserver.cc:460] Webserver started at http://127.13.157.131:44523/ using document root <none> and password file <none>
I20260925 02:43:36.271437 21957 fs_manager.cc:359] Metadata directory not provided
I20260925 02:43:36.271502 21957 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:43:36.271705 21957 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:43:36.273651 21957 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790304178325258-13942-0/minicluster-data/ts-2/data/instance:
uuid: "935a4168b81f4455b6bcf9ba18b1bf80"
format_stamp: "Formatted at 2026-09-25 02:43:36 on dist-test-slave-kznw"
server_key: "552a8b13da081c71a47eb0bf9775c366"
server_key_iv: "2b0f99c70098a7d2eb10e0e327c621f7"
server_key_version: "encryptionkey@0"
I20260925 02:43:36.274184 21957 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790304178325258-13942-0/minicluster-data/ts-2/wal/instance:
uuid: "935a4168b81f4455b6bcf9ba18b1bf80"
format_stamp: "Formatted at 2026-09-25 02:43:36 on dist-test-slave-kznw"
server_key: "552a8b13da081c71a47eb0bf9775c366"
server_key_iv: "2b0f99c70098a7d2eb10e0e327c621f7"
server_key_version: "encryptionkey@0"
I20260925 02:43:36.277845 21957 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260925 02:43:36.280228 21972 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:36.281526 21957 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260925 02:43:36.281639 21957 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790304178325258-13942-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790304178325258-13942-0/minicluster-data/ts-2/wal
uuid: "935a4168b81f4455b6bcf9ba18b1bf80"
format_stamp: "Formatted at 2026-09-25 02:43:36 on dist-test-slave-kznw"
server_key: "552a8b13da081c71a47eb0bf9775c366"
server_key_iv: "2b0f99c70098a7d2eb10e0e327c621f7"
server_key_version: "encryptionkey@0"
I20260925 02:43:36.281741 21957 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790304178325258-13942-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790304178325258-13942-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790304178325258-13942-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:43:36.297838 21957 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:43:36.300724 21957 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:43:36.300905 21957 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:43:36.302287 21957 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:43:36.302335 21957 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:36.302387 21957 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:43:36.302414 21957 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 25 02:43:36 dist-test-slave-kznw krb5kdc[21556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304216, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.157.131@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM
I20260925 02:43:36.312683 21957 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.131:41903
I20260925 02:43:36.312749 22089 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.131:41903 every 8 connection(s)
I20260925 02:43:36.313585 21957 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790304178325258-13942-0/minicluster-data/ts-2/data/info.pb
I20260925 02:43:36.315665 21651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:36.303002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.131:51179 (local address 127.13.157.190:43945)
0925 02:43:36.303145 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:36.303149 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:36.304193 (+ 1044us) server_negotiation.cc:407] Connection header received
0925 02:43:36.305258 (+ 1065us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:36.305261 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:36.305323 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:36.305379 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:36.306832 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:36.307456 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:36.308660 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:36.308929 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:36.309019 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:36.312014 (+ 2995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:36.312017 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:36.312019 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:43:36.312048 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:36.313535 (+ 1487us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:36.314120 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:36.314124 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:36.314126 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:36.314223 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:36.314582 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:36.314586 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:36.314588 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:36.314774 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:36.314864 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:36.315372 (+ 508us) server_negotiation.cc:299] Negotiation successful
0925 02:43:36.315499 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260925 02:43:36.316406 21981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:36.303524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:43945 (local address 127.13.157.131:51179)
0925 02:43:36.304055 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:36.304098 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:36.304998 (+ 900us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:36.305753 (+ 755us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:36.305763 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:36.306109 (+ 346us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:36.306668 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:36.306689 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:36.307584 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:36.307588 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:36.308537 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:36.308547 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:36.309106 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:36.309111 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:36.309302 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:36.309908 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:36.309929 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:36.311768 (+ 1839us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:43:36.313653 (+ 1885us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:36.313659 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:36.313674 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:36.314000 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:36.314326 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:36.314329 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:36.314331 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:36.314483 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:36.314878 (+ 395us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:43:36.314885 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:43:36.315144 (+ 259us) client_negotiation.cc:769] Sending connection context
0925 02:43:36.315307 (+ 163us) client_negotiation.cc:240] Negotiation successful
0925 02:43:36.315626 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":136,"threads_started":1}
I20260925 02:43:36.317451 22090 heartbeater.cc:347] Connected to a master server at 127.13.157.190:43945
I20260925 02:43:36.317665 22090 heartbeater.cc:464] Registering TS with master...
I20260925 02:43:36.318126 22090 heartbeater.cc:511] Master 127.13.157.190:43945 requested a full tablet report, sending...
I20260925 02:43:36.319124 21601 ts_manager.cc:194] Registered new tserver with Master: 935a4168b81f4455b6bcf9ba18b1bf80 (127.13.157.131:41903)
I20260925 02:43:36.319711 21601 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.157.131@KRBTEST.COM'} at 127.13.157.131:51179
I20260925 02:43:36.320192 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 21957
I20260925 02:43:36.320317 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790304178325258-13942-0/minicluster-data/ts-2/wal/instance
I20260925 02:43:36.320703 13942 external_mini_cluster.cc:1428] Setting key 7f00a139f022365b8e549a95bd5fe94c
I20260925 02:43:36.322592 13942 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 02:43:36.326745 21651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:36.319742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.131:46581 (local address 127.13.157.190:43945)
0925 02:43:36.319853 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:36.319857 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:36.319993 (+ 136us) server_negotiation.cc:407] Connection header received
0925 02:43:36.320133 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:36.320136 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:36.320183 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:36.320263 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:36.320749 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:36.321477 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:36.322448 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:36.322753 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:36.322846 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:36.323936 (+ 1090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:36.323939 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:36.323941 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:43:36.323962 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:36.325223 (+ 1261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:36.325649 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:36.325651 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:36.325653 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:36.325716 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:36.325976 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:36.325979 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:36.325981 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:36.326180 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:36.326265 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:36.326501 (+ 236us) server_negotiation.cc:299] Negotiation successful
0925 02:43:36.326597 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260925 02:43:36.326756 22096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:36.319655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:43945 (local address 127.13.157.131:46581)
0925 02:43:36.319912 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:36.319926 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:36.320042 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:36.320293 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:36.320296 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:36.320490 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:36.320645 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:36.320653 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:36.321618 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:36.321621 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:36.322344 (+ 723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:36.322355 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:36.322863 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:36.322867 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:36.322984 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:36.323440 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:36.323453 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:36.323816 (+ 363us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:43:36.325325 (+ 1509us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:36.325328 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:36.325330 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:36.325576 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:36.325804 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:36.325807 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:36.325810 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:36.325903 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:36.326272 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:43:36.326275 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:43:36.326356 (+ 81us) client_negotiation.cc:769] Sending connection context
0925 02:43:36.326438 (+ 82us) client_negotiation.cc:240] Negotiation successful
0925 02:43:36.326570 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":77,"threads_started":1}
Sep 25 02:43:36 dist-test-slave-kznw krb5kdc[21556](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304216, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 25 02:43:36 dist-test-slave-kznw krb5kdc[21556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.13.157.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 02:43:36 dist-test-slave-kznw krb5kdc[21556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.13.157.190@KRBTEST.COM, Server not found in Kerberos database
W20260925 02:43:36.345412 21651 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:43:36.339703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48202 (local address 127.13.157.190:43945)
0925 02:43:36.339828 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:36.339831 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:36.339843 (+ 12us) server_negotiation.cc:407] Connection header received
0925 02:43:36.339878 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:36.339880 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:36.339919 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:36.339981 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:36.340706 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:36.341283 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:36.342103 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:36.342311 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:36.342388 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:36.345296 (+ 2908us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48202: BlockingRecv error: recv got EOF from 127.0.0.1:48202 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Sep 25 02:43:36 dist-test-slave-kznw krb5kdc[21556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304216, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM
I20260925 02:43:36.354521 21651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:36.347031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48204 (local address 127.13.157.190:43945)
0925 02:43:36.347153 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:36.347157 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:36.347221 (+ 64us) server_negotiation.cc:407] Connection header received
0925 02:43:36.347343 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:36.347346 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:36.347385 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:36.347450 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:36.348009 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:36.348504 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:36.349255 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:36.349461 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:36.349552 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:36.351704 (+ 2152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:36.351707 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:36.351709 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:43:36.351731 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:36.352999 (+ 1268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:36.353419 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:36.353423 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:36.353425 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:36.353571 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:36.353797 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:36.353800 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:36.353801 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:36.353970 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:36.354133 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:36.354295 (+ 162us) server_negotiation.cc:299] Negotiation successful
0925 02:43:36.354398 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260925 02:43:36.357400 21601 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48204:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 02:43:36.360584 21601 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 02:43:36.374485 22113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:36.368912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35980 (local address 127.13.157.129:40507)
0925 02:43:36.369858 (+ 946us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:36.369873 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:36.369896 (+ 23us) server_negotiation.cc:407] Connection header received
0925 02:43:36.369975 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:36.369982 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:36.370259 (+ 277us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:36.370408 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:43:36.370688 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:36.371602 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:36.373045 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:36.373833 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:36.373961 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:36.374097 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:36.374258 (+ 161us) server_negotiation.cc:299] Negotiation successful
0925 02:43:36.374345 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":839,"thread_start_us":74,"threads_started":1}
I20260925 02:43:36.374476 22112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:36.368722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.129:40507 (local address 127.0.0.1:35980)
0925 02:43:36.369108 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:36.369160 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:36.369287 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:36.370407 (+ 1120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:36.370413 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:36.370437 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:43:36.370582 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:36.370589 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:36.371704 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:36.371707 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:36.372920 (+ 1213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:36.372931 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:36.373936 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:36.373940 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:36.374063 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:36.374101 (+ 38us) client_negotiation.cc:769] Sending connection context
0925 02:43:36.374202 (+ 101us) client_negotiation.cc:240] Negotiation successful
0925 02:43:36.374319 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"mutex_wait_us":18,"thread_start_us":106,"threads_started":1}
I20260925 02:43:36.374991 22115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:36.368966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.130:33705 (local address 127.0.0.1:42642)
0925 02:43:36.369734 (+ 768us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:36.369746 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:36.369829 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:36.370509 (+ 680us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:36.370512 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:36.370540 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:43:36.370635 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:36.370640 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:36.372195 (+ 1555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:36.372199 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:36.373280 (+ 1081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:36.373289 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:36.374534 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:36.374537 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:36.374627 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:36.374680 (+ 53us) client_negotiation.cc:769] Sending connection context
0925 02:43:36.374749 (+ 69us) client_negotiation.cc:240] Negotiation successful
0925 02:43:36.374816 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":708,"mutex_wait_us":83,"thread_start_us":98,"threads_started":1}
I20260925 02:43:36.375046 22116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:36.369534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42642 (local address 127.13.157.130:33705)
0925 02:43:36.370070 (+ 536us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:36.370082 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:36.370104 (+ 22us) server_negotiation.cc:407] Connection header received
0925 02:43:36.370178 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:36.370184 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:36.370372 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:36.370772 (+ 400us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:43:36.370924 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:36.372081 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:36.373385 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:36.374130 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:36.374337 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:36.374466 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:36.374790 (+ 324us) server_negotiation.cc:299] Negotiation successful
0925 02:43:36.374885 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":430,"thread_start_us":105,"threads_started":1}
I20260925 02:43:36.375216 22117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:36.369712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43284 (local address 127.13.157.131:41903)
0925 02:43:36.370113 (+ 401us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:36.370123 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:36.370488 (+ 365us) server_negotiation.cc:407] Connection header received
0925 02:43:36.370564 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:36.370569 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:36.370735 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:36.370873 (+ 138us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:43:36.371112 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:36.372206 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:36.373597 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:36.374352 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:36.374473 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:36.374579 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:36.374981 (+ 402us) server_negotiation.cc:299] Negotiation successful
0925 02:43:36.375083 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":98,"threads_started":1}
I20260925 02:43:36.375430 22114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:36.369490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.131:41903 (local address 127.0.0.1:43284)
0925 02:43:36.370307 (+ 817us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:36.370319 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:36.370400 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:36.370865 (+ 465us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:36.370867 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:36.370886 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:43:36.371005 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:36.371011 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:36.372319 (+ 1308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:36.372322 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:36.373474 (+ 1152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:36.373485 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:36.374631 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:36.374634 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:36.374723 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:36.374752 (+ 29us) client_negotiation.cc:769] Sending connection context
0925 02:43:36.374807 (+ 55us) client_negotiation.cc:240] Negotiation successful
0925 02:43:36.374857 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":762,"mutex_wait_us":3,"thread_start_us":89,"threads_started":1}
I20260925 02:43:36.377758 21740 tablet_service.cc:1467] Processing CreateTablet for tablet 64a2abfbedf54616b212772c616114b2 (DEFAULT_TABLE table=test-table [id=eac74822df6c424fadd665e2b53f2ef7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:43:36.378118 22023 tablet_service.cc:1467] Processing CreateTablet for tablet 64a2abfbedf54616b212772c616114b2 (DEFAULT_TABLE table=test-table [id=eac74822df6c424fadd665e2b53f2ef7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:43:36.378118 21882 tablet_service.cc:1467] Processing CreateTablet for tablet 64a2abfbedf54616b212772c616114b2 (DEFAULT_TABLE table=test-table [id=eac74822df6c424fadd665e2b53f2ef7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:43:36.378726 21740 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 64a2abfbedf54616b212772c616114b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:43:36.379163 21882 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 64a2abfbedf54616b212772c616114b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:43:36.379170 22023 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 64a2abfbedf54616b212772c616114b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:43:36.385336 22118 tablet_bootstrap.cc:492] T 64a2abfbedf54616b212772c616114b2 P 77e5b44255714c0db6e1fa4e7130c90f: Bootstrap starting.
I20260925 02:43:36.387233 22120 tablet_bootstrap.cc:492] T 64a2abfbedf54616b212772c616114b2 P 935a4168b81f4455b6bcf9ba18b1bf80: Bootstrap starting.
I20260925 02:43:36.388221 22118 tablet_bootstrap.cc:654] T 64a2abfbedf54616b212772c616114b2 P 77e5b44255714c0db6e1fa4e7130c90f: Neither blocks nor log segments found. Creating new log.
I20260925 02:43:36.389084 22118 log.cc:824] T 64a2abfbedf54616b212772c616114b2 P 77e5b44255714c0db6e1fa4e7130c90f: Log is configured to *not* fsync() on all Append() calls
I20260925 02:43:36.389314 22120 tablet_bootstrap.cc:654] T 64a2abfbedf54616b212772c616114b2 P 935a4168b81f4455b6bcf9ba18b1bf80: Neither blocks nor log segments found. Creating new log.
I20260925 02:43:36.389801 22119 tablet_bootstrap.cc:492] T 64a2abfbedf54616b212772c616114b2 P eb7b2024cae043fd8827ea003fbbcdbf: Bootstrap starting.
I20260925 02:43:36.390132 22120 log.cc:824] T 64a2abfbedf54616b212772c616114b2 P 935a4168b81f4455b6bcf9ba18b1bf80: Log is configured to *not* fsync() on all Append() calls
I20260925 02:43:36.391819 22119 tablet_bootstrap.cc:654] T 64a2abfbedf54616b212772c616114b2 P eb7b2024cae043fd8827ea003fbbcdbf: Neither blocks nor log segments found. Creating new log.
I20260925 02:43:36.392089 22118 tablet_bootstrap.cc:492] T 64a2abfbedf54616b212772c616114b2 P 77e5b44255714c0db6e1fa4e7130c90f: No bootstrap required, opened a new log
I20260925 02:43:36.392185 22120 tablet_bootstrap.cc:492] T 64a2abfbedf54616b212772c616114b2 P 935a4168b81f4455b6bcf9ba18b1bf80: No bootstrap required, opened a new log
I20260925 02:43:36.392279 22118 ts_tablet_manager.cc:1397] T 64a2abfbedf54616b212772c616114b2 P 77e5b44255714c0db6e1fa4e7130c90f: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260925 02:43:36.392364 22120 ts_tablet_manager.cc:1397] T 64a2abfbedf54616b212772c616114b2 P 935a4168b81f4455b6bcf9ba18b1bf80: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260925 02:43:36.392607 22119 log.cc:824] T 64a2abfbedf54616b212772c616114b2 P eb7b2024cae043fd8827ea003fbbcdbf: Log is configured to *not* fsync() on all Append() calls
I20260925 02:43:36.394845 22119 tablet_bootstrap.cc:492] T 64a2abfbedf54616b212772c616114b2 P eb7b2024cae043fd8827ea003fbbcdbf: No bootstrap required, opened a new log
I20260925 02:43:36.395113 22119 ts_tablet_manager.cc:1397] T 64a2abfbedf54616b212772c616114b2 P eb7b2024cae043fd8827ea003fbbcdbf: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260925 02:43:36.395715 22118 raft_consensus.cc:348] T 64a2abfbedf54616b212772c616114b2 P 77e5b44255714c0db6e1fa4e7130c90f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77e5b44255714c0db6e1fa4e7130c90f" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 40507 } } peers { permanent_uuid: "935a4168b81f4455b6bcf9ba18b1bf80" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 41903 } } peers { permanent_uuid: "eb7b2024cae043fd8827ea003fbbcdbf" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33705 } }
I20260925 02:43:36.395921 22118 raft_consensus.cc:374] T 64a2abfbedf54616b212772c616114b2 P 77e5b44255714c0db6e1fa4e7130c90f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:43:36.395972 22118 raft_consensus.cc:729] T 64a2abfbedf54616b212772c616114b2 P 77e5b44255714c0db6e1fa4e7130c90f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77e5b44255714c0db6e1fa4e7130c90f, State: Initialized, Role: FOLLOWER
I20260925 02:43:36.396021 22120 raft_consensus.cc:348] T 64a2abfbedf54616b212772c616114b2 P 935a4168b81f4455b6bcf9ba18b1bf80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77e5b44255714c0db6e1fa4e7130c90f" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 40507 } } peers { permanent_uuid: "935a4168b81f4455b6bcf9ba18b1bf80" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 41903 } } peers { permanent_uuid: "eb7b2024cae043fd8827ea003fbbcdbf" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33705 } }
I20260925 02:43:36.396262 22120 raft_consensus.cc:374] T 64a2abfbedf54616b212772c616114b2 P 935a4168b81f4455b6bcf9ba18b1bf80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:43:36.396317 22120 raft_consensus.cc:729] T 64a2abfbedf54616b212772c616114b2 P 935a4168b81f4455b6bcf9ba18b1bf80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 935a4168b81f4455b6bcf9ba18b1bf80, State: Initialized, Role: FOLLOWER
I20260925 02:43:36.396400 22118 consensus_queue.cc:260] T 64a2abfbedf54616b212772c616114b2 P 77e5b44255714c0db6e1fa4e7130c90f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77e5b44255714c0db6e1fa4e7130c90f" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 40507 } } peers { permanent_uuid: "935a4168b81f4455b6bcf9ba18b1bf80" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 41903 } } peers { permanent_uuid: "eb7b2024cae043fd8827ea003fbbcdbf" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33705 } }
I20260925 02:43:36.396976 21807 heartbeater.cc:503] Master 127.13.157.190:43945 was elected leader, sending a full tablet report...
I20260925 02:43:36.397318 22118 ts_tablet_manager.cc:1428] T 64a2abfbedf54616b212772c616114b2 P 77e5b44255714c0db6e1fa4e7130c90f: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260925 02:43:36.396739 22120 consensus_queue.cc:260] T 64a2abfbedf54616b212772c616114b2 P 935a4168b81f4455b6bcf9ba18b1bf80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77e5b44255714c0db6e1fa4e7130c90f" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 40507 } } peers { permanent_uuid: "935a4168b81f4455b6bcf9ba18b1bf80" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 41903 } } peers { permanent_uuid: "eb7b2024cae043fd8827ea003fbbcdbf" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33705 } }
I20260925 02:43:36.398478 22090 heartbeater.cc:503] Master 127.13.157.190:43945 was elected leader, sending a full tablet report...
I20260925 02:43:36.398797 22120 ts_tablet_manager.cc:1428] T 64a2abfbedf54616b212772c616114b2 P 935a4168b81f4455b6bcf9ba18b1bf80: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260925 02:43:36.398752 22119 raft_consensus.cc:348] T 64a2abfbedf54616b212772c616114b2 P eb7b2024cae043fd8827ea003fbbcdbf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77e5b44255714c0db6e1fa4e7130c90f" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 40507 } } peers { permanent_uuid: "935a4168b81f4455b6bcf9ba18b1bf80" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 41903 } } peers { permanent_uuid: "eb7b2024cae043fd8827ea003fbbcdbf" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33705 } }
I20260925 02:43:36.398993 22119 raft_consensus.cc:374] T 64a2abfbedf54616b212772c616114b2 P eb7b2024cae043fd8827ea003fbbcdbf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:43:36.399051 22119 raft_consensus.cc:729] T 64a2abfbedf54616b212772c616114b2 P eb7b2024cae043fd8827ea003fbbcdbf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb7b2024cae043fd8827ea003fbbcdbf, State: Initialized, Role: FOLLOWER
I20260925 02:43:36.399406 22119 consensus_queue.cc:260] T 64a2abfbedf54616b212772c616114b2 P eb7b2024cae043fd8827ea003fbbcdbf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77e5b44255714c0db6e1fa4e7130c90f" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 40507 } } peers { permanent_uuid: "935a4168b81f4455b6bcf9ba18b1bf80" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 41903 } } peers { permanent_uuid: "eb7b2024cae043fd8827ea003fbbcdbf" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33705 } }
I20260925 02:43:36.399981 21948 heartbeater.cc:503] Master 127.13.157.190:43945 was elected leader, sending a full tablet report...
I20260925 02:43:36.400303 22119 ts_tablet_manager.cc:1428] T 64a2abfbedf54616b212772c616114b2 P eb7b2024cae043fd8827ea003fbbcdbf: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260925 02:43:36.424019 22125 raft_consensus.cc:482] T 64a2abfbedf54616b212772c616114b2 P 935a4168b81f4455b6bcf9ba18b1bf80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 02:43:36.424209 22125 raft_consensus.cc:504] T 64a2abfbedf54616b212772c616114b2 P 935a4168b81f4455b6bcf9ba18b1bf80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77e5b44255714c0db6e1fa4e7130c90f" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 40507 } } peers { permanent_uuid: "935a4168b81f4455b6bcf9ba18b1bf80" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 41903 } } peers { permanent_uuid: "eb7b2024cae043fd8827ea003fbbcdbf" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33705 } }
I20260925 02:43:36.425406 22125 leader_election.cc:288] T 64a2abfbedf54616b212772c616114b2 P 935a4168b81f4455b6bcf9ba18b1bf80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 77e5b44255714c0db6e1fa4e7130c90f (127.13.157.129:40507), eb7b2024cae043fd8827ea003fbbcdbf (127.13.157.130:33705)
I20260925 02:43:36.428856 21981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:36.425617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.129:40507 (local address 127.13.157.131:35825)
0925 02:43:36.425809 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:36.425823 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:36.425910 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:36.426199 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:36.426201 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:36.426220 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:43:36.426319 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:36.426324 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:36.427060 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:36.427064 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:36.427828 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:36.427836 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:36.428430 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:36.428434 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:36.428546 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:36.428567 (+ 21us) client_negotiation.cc:769] Sending connection context
0925 02:43:36.428650 (+ 83us) client_negotiation.cc:240] Negotiation successful
0925 02:43:36.428723 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"mutex_wait_us":5}
I20260925 02:43:36.428856 22113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:36.425733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.131:35825 (local address 127.13.157.129:40507)
0925 02:43:36.425848 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:36.425853 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:36.425870 (+ 17us) server_negotiation.cc:407] Connection header received
0925 02:43:36.426048 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:36.426052 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:36.426103 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:36.426167 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:43:36.426420 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:36.426957 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:36.427947 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:36.428325 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:36.428428 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:36.428492 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:36.428671 (+ 179us) server_negotiation.cc:299] Negotiation successful
0925 02:43:36.428741 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260925 02:43:36.429550 22116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:36.425802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.131:60717 (local address 127.13.157.130:33705)
0925 02:43:36.425909 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:36.425913 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:36.426168 (+ 255us) server_negotiation.cc:407] Connection header received
0925 02:43:36.426279 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:36.426282 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:36.426323 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:36.426383 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:43:36.426640 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:36.427181 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:36.428517 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:36.429032 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:36.429147 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:36.429194 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:36.429403 (+ 209us) server_negotiation.cc:299] Negotiation successful
0925 02:43:36.429458 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260925 02:43:36.429600 22127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:36.425641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.130:33705 (local address 127.13.157.131:60717)
0925 02:43:36.426084 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:36.426097 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:36.426196 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:36.426404 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:36.426407 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:36.426421 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:43:36.426546 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:36.426551 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:36.427275 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:36.427278 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:36.428391 (+ 1113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:36.428400 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:36.429162 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:36.429166 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:36.429261 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:36.429310 (+ 49us) client_negotiation.cc:769] Sending connection context
0925 02:43:36.429386 (+ 76us) client_negotiation.cc:240] Negotiation successful
0925 02:43:36.429467 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":130,"threads_started":1}
I20260925 02:43:36.429569 21761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64a2abfbedf54616b212772c616114b2" candidate_uuid: "935a4168b81f4455b6bcf9ba18b1bf80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "77e5b44255714c0db6e1fa4e7130c90f" is_pre_election: true
I20260925 02:43:36.429893 21761 raft_consensus.cc:2454] T 64a2abfbedf54616b212772c616114b2 P 77e5b44255714c0db6e1fa4e7130c90f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 935a4168b81f4455b6bcf9ba18b1bf80 in term 0.
I20260925 02:43:36.430142 21902 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64a2abfbedf54616b212772c616114b2" candidate_uuid: "935a4168b81f4455b6bcf9ba18b1bf80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb7b2024cae043fd8827ea003fbbcdbf" is_pre_election: true
I20260925 02:43:36.430399 21902 raft_consensus.cc:2454] T 64a2abfbedf54616b212772c616114b2 P eb7b2024cae043fd8827ea003fbbcdbf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 935a4168b81f4455b6bcf9ba18b1bf80 in term 0.
I20260925 02:43:36.430419 21974 leader_election.cc:302] T 64a2abfbedf54616b212772c616114b2 P 935a4168b81f4455b6bcf9ba18b1bf80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 77e5b44255714c0db6e1fa4e7130c90f, 935a4168b81f4455b6bcf9ba18b1bf80; no voters:
I20260925 02:43:36.430696 22125 raft_consensus.cc:2781] T 64a2abfbedf54616b212772c616114b2 P 935a4168b81f4455b6bcf9ba18b1bf80 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 02:43:36.430789 22125 raft_consensus.cc:482] T 64a2abfbedf54616b212772c616114b2 P 935a4168b81f4455b6bcf9ba18b1bf80 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 02:43:36.430822 22125 raft_consensus.cc:3037] T 64a2abfbedf54616b212772c616114b2 P 935a4168b81f4455b6bcf9ba18b1bf80 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:43:36.431895 22125 raft_consensus.cc:504] T 64a2abfbedf54616b212772c616114b2 P 935a4168b81f4455b6bcf9ba18b1bf80 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77e5b44255714c0db6e1fa4e7130c90f" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 40507 } } peers { permanent_uuid: "935a4168b81f4455b6bcf9ba18b1bf80" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 41903 } } peers { permanent_uuid: "eb7b2024cae043fd8827ea003fbbcdbf" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33705 } }
I20260925 02:43:36.432248 22125 leader_election.cc:288] T 64a2abfbedf54616b212772c616114b2 P 935a4168b81f4455b6bcf9ba18b1bf80 [CANDIDATE]: Term 1 election: Requested vote from peers 77e5b44255714c0db6e1fa4e7130c90f (127.13.157.129:40507), eb7b2024cae043fd8827ea003fbbcdbf (127.13.157.130:33705)
I20260925 02:43:36.432534 21902 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64a2abfbedf54616b212772c616114b2" candidate_uuid: "935a4168b81f4455b6bcf9ba18b1bf80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb7b2024cae043fd8827ea003fbbcdbf"
I20260925 02:43:36.432546 21761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64a2abfbedf54616b212772c616114b2" candidate_uuid: "935a4168b81f4455b6bcf9ba18b1bf80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "77e5b44255714c0db6e1fa4e7130c90f"
I20260925 02:43:36.432655 21902 raft_consensus.cc:3037] T 64a2abfbedf54616b212772c616114b2 P eb7b2024cae043fd8827ea003fbbcdbf [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:43:36.432687 21761 raft_consensus.cc:3037] T 64a2abfbedf54616b212772c616114b2 P 77e5b44255714c0db6e1fa4e7130c90f [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:43:36.433617 21761 raft_consensus.cc:2454] T 64a2abfbedf54616b212772c616114b2 P 77e5b44255714c0db6e1fa4e7130c90f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 935a4168b81f4455b6bcf9ba18b1bf80 in term 1.
I20260925 02:43:36.433734 21902 raft_consensus.cc:2454] T 64a2abfbedf54616b212772c616114b2 P eb7b2024cae043fd8827ea003fbbcdbf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 935a4168b81f4455b6bcf9ba18b1bf80 in term 1.
I20260925 02:43:36.433883 21974 leader_election.cc:302] T 64a2abfbedf54616b212772c616114b2 P 935a4168b81f4455b6bcf9ba18b1bf80 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 77e5b44255714c0db6e1fa4e7130c90f, 935a4168b81f4455b6bcf9ba18b1bf80; no voters:
I20260925 02:43:36.434038 22125 raft_consensus.cc:2781] T 64a2abfbedf54616b212772c616114b2 P 935a4168b81f4455b6bcf9ba18b1bf80 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:43:36.434242 22125 raft_consensus.cc:686] T 64a2abfbedf54616b212772c616114b2 P 935a4168b81f4455b6bcf9ba18b1bf80 [term 1 LEADER]: Becoming Leader. State: Replica: 935a4168b81f4455b6bcf9ba18b1bf80, State: Running, Role: LEADER
I20260925 02:43:36.434540 22125 consensus_queue.cc:237] T 64a2abfbedf54616b212772c616114b2 P 935a4168b81f4455b6bcf9ba18b1bf80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77e5b44255714c0db6e1fa4e7130c90f" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 40507 } } peers { permanent_uuid: "935a4168b81f4455b6bcf9ba18b1bf80" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 41903 } } peers { permanent_uuid: "eb7b2024cae043fd8827ea003fbbcdbf" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33705 } }
I20260925 02:43:36.437697 21598 catalog_manager.cc:5495] T 64a2abfbedf54616b212772c616114b2 P 935a4168b81f4455b6bcf9ba18b1bf80 reported cstate change: term changed from 0 to 1, leader changed from <none> to 935a4168b81f4455b6bcf9ba18b1bf80 (127.13.157.131). New cstate: current_term: 1 leader_uuid: "935a4168b81f4455b6bcf9ba18b1bf80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77e5b44255714c0db6e1fa4e7130c90f" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 40507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "935a4168b81f4455b6bcf9ba18b1bf80" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 41903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb7b2024cae043fd8827ea003fbbcdbf" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33705 } health_report { overall_health: UNKNOWN } } }
I20260925 02:43:36.455540 22117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:36.452254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43286 (local address 127.13.157.131:41903)
0925 02:43:36.452381 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:36.452384 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:36.452397 (+ 13us) server_negotiation.cc:407] Connection header received
0925 02:43:36.452524 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:36.452527 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:36.452575 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:36.452659 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 02:43:36.452834 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:36.453361 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:36.454213 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:36.454433 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:36.454524 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:36.454737 (+ 213us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 02:43:36.455205 (+ 468us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 02:43:36.455288 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:36.455385 (+ 97us) server_negotiation.cc:299] Negotiation successful
0925 02:43:36.455434 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260925 02:43:36.460983 21902 raft_consensus.cc:1261] T 64a2abfbedf54616b212772c616114b2 P eb7b2024cae043fd8827ea003fbbcdbf [term 1 FOLLOWER]: Refusing update from remote peer 935a4168b81f4455b6bcf9ba18b1bf80: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:43:36.460983 21761 raft_consensus.cc:1261] T 64a2abfbedf54616b212772c616114b2 P 77e5b44255714c0db6e1fa4e7130c90f [term 1 FOLLOWER]: Refusing update from remote peer 935a4168b81f4455b6bcf9ba18b1bf80: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:43:36.461661 22128 consensus_queue.cc:1035] T 64a2abfbedf54616b212772c616114b2 P 935a4168b81f4455b6bcf9ba18b1bf80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb7b2024cae043fd8827ea003fbbcdbf" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:43:36.461831 22125 consensus_queue.cc:1035] T 64a2abfbedf54616b212772c616114b2 P 935a4168b81f4455b6bcf9ba18b1bf80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "77e5b44255714c0db6e1fa4e7130c90f" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 40507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:43:36.469053 22133 mvcc.cc:205] Tried to move back new op lower bound from 7333086070617530368 to 7333086070518718464. Current Snapshot: MvccSnapshot[applied={T|T < 7333086070617530368}]
I20260925 02:43:36.476207 21598 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48204:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 02:43:36.476361 21598 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48204:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 02:43:36.478585 21598 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9120630de98a487eb499476860ce3215: Sending DeleteTablet for 3 replicas of tablet 64a2abfbedf54616b212772c616114b2
I20260925 02:43:36.479298 21882 tablet_service.cc:1514] Processing DeleteTablet for tablet 64a2abfbedf54616b212772c616114b2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:43:36 UTC) from {username='oryx', principal='oryx/127.13.157.190@KRBTEST.COM'} at 127.0.0.1:42642
I20260925 02:43:36.479377 21740 tablet_service.cc:1514] Processing DeleteTablet for tablet 64a2abfbedf54616b212772c616114b2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:43:36 UTC) from {username='oryx', principal='oryx/127.13.157.190@KRBTEST.COM'} at 127.0.0.1:35980
I20260925 02:43:36.479789 22023 tablet_service.cc:1514] Processing DeleteTablet for tablet 64a2abfbedf54616b212772c616114b2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:43:36 UTC) from {username='oryx', principal='oryx/127.13.157.190@KRBTEST.COM'} at 127.0.0.1:43284
I20260925 02:43:36.479825 22142 tablet_replica.cc:330] T 64a2abfbedf54616b212772c616114b2 P eb7b2024cae043fd8827ea003fbbcdbf: stopping tablet replica
I20260925 02:43:36.480150 22142 raft_consensus.cc:2229] T 64a2abfbedf54616b212772c616114b2 P eb7b2024cae043fd8827ea003fbbcdbf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 02:43:36.480185 22143 tablet_replica.cc:330] T 64a2abfbedf54616b212772c616114b2 P 935a4168b81f4455b6bcf9ba18b1bf80: stopping tablet replica
I20260925 02:43:36.480464 22142 raft_consensus.cc:2258] T 64a2abfbedf54616b212772c616114b2 P eb7b2024cae043fd8827ea003fbbcdbf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:43:36.480618 22143 raft_consensus.cc:2229] T 64a2abfbedf54616b212772c616114b2 P 935a4168b81f4455b6bcf9ba18b1bf80 [term 1 LEADER]: Raft consensus shutting down.
I20260925 02:43:36.480821 22141 tablet_replica.cc:330] T 64a2abfbedf54616b212772c616114b2 P 77e5b44255714c0db6e1fa4e7130c90f: stopping tablet replica
I20260925 02:43:36.481068 22143 raft_consensus.cc:2258] T 64a2abfbedf54616b212772c616114b2 P 935a4168b81f4455b6bcf9ba18b1bf80 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:43:36.481024 21598 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48204:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 02:43:36.481109 22141 raft_consensus.cc:2229] T 64a2abfbedf54616b212772c616114b2 P 77e5b44255714c0db6e1fa4e7130c90f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 02:43:36.481375 22141 raft_consensus.cc:2258] T 64a2abfbedf54616b212772c616114b2 P 77e5b44255714c0db6e1fa4e7130c90f [term 1 FOLLOWER]: Raft consensus is shut down!
W20260925 02:43:36.481410 21598 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 02:43:36.482863 22143 ts_tablet_manager.cc:1905] T 64a2abfbedf54616b212772c616114b2 P 935a4168b81f4455b6bcf9ba18b1bf80: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:43:36.485400 22142 ts_tablet_manager.cc:1905] T 64a2abfbedf54616b212772c616114b2 P eb7b2024cae043fd8827ea003fbbcdbf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:43:36.485982 22023 tablet_service.cc:1467] Processing CreateTablet for tablet dc5e802aa3034eab8836b5934f869fba (DEFAULT_TABLE table=test-table [id=ff3a161f2faf427fb8f51126955f0d1f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:43:36.486235 22023 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dc5e802aa3034eab8836b5934f869fba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:43:36.485951 21882 tablet_service.cc:1467] Processing CreateTablet for tablet dc5e802aa3034eab8836b5934f869fba (DEFAULT_TABLE table=test-table [id=ff3a161f2faf427fb8f51126955f0d1f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:43:36.486430 21882 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dc5e802aa3034eab8836b5934f869fba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:43:36.488273 22120 tablet_bootstrap.cc:492] T dc5e802aa3034eab8836b5934f869fba P 935a4168b81f4455b6bcf9ba18b1bf80: Bootstrap starting.
I20260925 02:43:36.488456 21740 tablet_service.cc:1467] Processing CreateTablet for tablet dc5e802aa3034eab8836b5934f869fba (DEFAULT_TABLE table=test-table [id=ff3a161f2faf427fb8f51126955f0d1f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:43:36.488718 21740 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dc5e802aa3034eab8836b5934f869fba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:43:36.488915 22119 tablet_bootstrap.cc:492] T dc5e802aa3034eab8836b5934f869fba P eb7b2024cae043fd8827ea003fbbcdbf: Bootstrap starting.
I20260925 02:43:36.489219 22120 tablet_bootstrap.cc:654] T dc5e802aa3034eab8836b5934f869fba P 935a4168b81f4455b6bcf9ba18b1bf80: Neither blocks nor log segments found. Creating new log.
I20260925 02:43:36.489807 22119 tablet_bootstrap.cc:654] T dc5e802aa3034eab8836b5934f869fba P eb7b2024cae043fd8827ea003fbbcdbf: Neither blocks nor log segments found. Creating new log.
I20260925 02:43:36.489987 22142 ts_tablet_manager.cc:1918] T 64a2abfbedf54616b212772c616114b2 P eb7b2024cae043fd8827ea003fbbcdbf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 02:43:36.491226 22118 tablet_bootstrap.cc:492] T dc5e802aa3034eab8836b5934f869fba P 77e5b44255714c0db6e1fa4e7130c90f: Bootstrap starting.
I20260925 02:43:36.491369 22142 log.cc:1196] T 64a2abfbedf54616b212772c616114b2 P eb7b2024cae043fd8827ea003fbbcdbf: Deleting WAL directory at /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790304178325258-13942-0/minicluster-data/ts-1/wal/wals/64a2abfbedf54616b212772c616114b2
I20260925 02:43:36.491727 22142 ts_tablet_manager.cc:1939] T 64a2abfbedf54616b212772c616114b2 P eb7b2024cae043fd8827ea003fbbcdbf: Deleting consensus metadata
I20260925 02:43:36.492000 22119 tablet_bootstrap.cc:492] T dc5e802aa3034eab8836b5934f869fba P eb7b2024cae043fd8827ea003fbbcdbf: No bootstrap required, opened a new log
I20260925 02:43:36.492089 22119 ts_tablet_manager.cc:1397] T dc5e802aa3034eab8836b5934f869fba P eb7b2024cae043fd8827ea003fbbcdbf: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260925 02:43:36.492127 22118 tablet_bootstrap.cc:654] T dc5e802aa3034eab8836b5934f869fba P 77e5b44255714c0db6e1fa4e7130c90f: Neither blocks nor log segments found. Creating new log.
I20260925 02:43:36.492550 22119 raft_consensus.cc:348] T dc5e802aa3034eab8836b5934f869fba P eb7b2024cae043fd8827ea003fbbcdbf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77e5b44255714c0db6e1fa4e7130c90f" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 40507 } } peers { permanent_uuid: "eb7b2024cae043fd8827ea003fbbcdbf" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33705 } } peers { permanent_uuid: "935a4168b81f4455b6bcf9ba18b1bf80" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 41903 } }
I20260925 02:43:36.492666 22119 raft_consensus.cc:374] T dc5e802aa3034eab8836b5934f869fba P eb7b2024cae043fd8827ea003fbbcdbf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:43:36.492705 22119 raft_consensus.cc:729] T dc5e802aa3034eab8836b5934f869fba P eb7b2024cae043fd8827ea003fbbcdbf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb7b2024cae043fd8827ea003fbbcdbf, State: Initialized, Role: FOLLOWER
I20260925 02:43:36.492794 22141 ts_tablet_manager.cc:1905] T 64a2abfbedf54616b212772c616114b2 P 77e5b44255714c0db6e1fa4e7130c90f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:43:36.492822 22119 consensus_queue.cc:260] T dc5e802aa3034eab8836b5934f869fba P eb7b2024cae043fd8827ea003fbbcdbf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77e5b44255714c0db6e1fa4e7130c90f" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 40507 } } peers { permanent_uuid: "eb7b2024cae043fd8827ea003fbbcdbf" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33705 } } peers { permanent_uuid: "935a4168b81f4455b6bcf9ba18b1bf80" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 41903 } }
I20260925 02:43:36.493089 22119 ts_tablet_manager.cc:1428] T dc5e802aa3034eab8836b5934f869fba P eb7b2024cae043fd8827ea003fbbcdbf: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260925 02:43:36.493312 22143 ts_tablet_manager.cc:1918] T 64a2abfbedf54616b212772c616114b2 P 935a4168b81f4455b6bcf9ba18b1bf80: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 02:43:36.493407 22143 log.cc:1196] T 64a2abfbedf54616b212772c616114b2 P 935a4168b81f4455b6bcf9ba18b1bf80: Deleting WAL directory at /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790304178325258-13942-0/minicluster-data/ts-2/wal/wals/64a2abfbedf54616b212772c616114b2
I20260925 02:43:36.493697 22143 ts_tablet_manager.cc:1939] T 64a2abfbedf54616b212772c616114b2 P 935a4168b81f4455b6bcf9ba18b1bf80: Deleting consensus metadata
I20260925 02:43:36.494233 21588 catalog_manager.cc:4841] TS eb7b2024cae043fd8827ea003fbbcdbf (127.13.157.130:33705): tablet 64a2abfbedf54616b212772c616114b2 (table test-table [id=eac74822df6c424fadd665e2b53f2ef7]) successfully deleted
I20260925 02:43:36.495106 21586 catalog_manager.cc:4841] TS 935a4168b81f4455b6bcf9ba18b1bf80 (127.13.157.131:41903): tablet 64a2abfbedf54616b212772c616114b2 (table test-table [id=eac74822df6c424fadd665e2b53f2ef7]) successfully deleted
I20260925 02:43:36.495705 22118 tablet_bootstrap.cc:492] T dc5e802aa3034eab8836b5934f869fba P 77e5b44255714c0db6e1fa4e7130c90f: No bootstrap required, opened a new log
I20260925 02:43:36.495790 22118 ts_tablet_manager.cc:1397] T dc5e802aa3034eab8836b5934f869fba P 77e5b44255714c0db6e1fa4e7130c90f: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260925 02:43:36.495894 22141 ts_tablet_manager.cc:1918] T 64a2abfbedf54616b212772c616114b2 P 77e5b44255714c0db6e1fa4e7130c90f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 02:43:36.495987 22141 log.cc:1196] T 64a2abfbedf54616b212772c616114b2 P 77e5b44255714c0db6e1fa4e7130c90f: Deleting WAL directory at /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790304178325258-13942-0/minicluster-data/ts-0/wal/wals/64a2abfbedf54616b212772c616114b2
I20260925 02:43:36.496270 22141 ts_tablet_manager.cc:1939] T 64a2abfbedf54616b212772c616114b2 P 77e5b44255714c0db6e1fa4e7130c90f: Deleting consensus metadata
I20260925 02:43:36.496474 22118 raft_consensus.cc:348] T dc5e802aa3034eab8836b5934f869fba P 77e5b44255714c0db6e1fa4e7130c90f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77e5b44255714c0db6e1fa4e7130c90f" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 40507 } } peers { permanent_uuid: "eb7b2024cae043fd8827ea003fbbcdbf" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33705 } } peers { permanent_uuid: "935a4168b81f4455b6bcf9ba18b1bf80" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 41903 } }
I20260925 02:43:36.496564 22118 raft_consensus.cc:374] T dc5e802aa3034eab8836b5934f869fba P 77e5b44255714c0db6e1fa4e7130c90f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:43:36.496599 22118 raft_consensus.cc:729] T dc5e802aa3034eab8836b5934f869fba P 77e5b44255714c0db6e1fa4e7130c90f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77e5b44255714c0db6e1fa4e7130c90f, State: Initialized, Role: FOLLOWER
I20260925 02:43:36.496718 22118 consensus_queue.cc:260] T dc5e802aa3034eab8836b5934f869fba P 77e5b44255714c0db6e1fa4e7130c90f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77e5b44255714c0db6e1fa4e7130c90f" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 40507 } } peers { permanent_uuid: "eb7b2024cae043fd8827ea003fbbcdbf" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33705 } } peers { permanent_uuid: "935a4168b81f4455b6bcf9ba18b1bf80" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 41903 } }
I20260925 02:43:36.496944 22118 ts_tablet_manager.cc:1428] T dc5e802aa3034eab8836b5934f869fba P 77e5b44255714c0db6e1fa4e7130c90f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260925 02:43:36.497251 21586 catalog_manager.cc:4841] TS 77e5b44255714c0db6e1fa4e7130c90f (127.13.157.129:40507): tablet 64a2abfbedf54616b212772c616114b2 (table test-table [id=eac74822df6c424fadd665e2b53f2ef7]) successfully deleted
I20260925 02:43:36.497757 22120 tablet_bootstrap.cc:492] T dc5e802aa3034eab8836b5934f869fba P 935a4168b81f4455b6bcf9ba18b1bf80: No bootstrap required, opened a new log
I20260925 02:43:36.497839 22120 ts_tablet_manager.cc:1397] T dc5e802aa3034eab8836b5934f869fba P 935a4168b81f4455b6bcf9ba18b1bf80: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20260925 02:43:36.498257 22120 raft_consensus.cc:348] T dc5e802aa3034eab8836b5934f869fba P 935a4168b81f4455b6bcf9ba18b1bf80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77e5b44255714c0db6e1fa4e7130c90f" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 40507 } } peers { permanent_uuid: "eb7b2024cae043fd8827ea003fbbcdbf" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33705 } } peers { permanent_uuid: "935a4168b81f4455b6bcf9ba18b1bf80" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 41903 } }
I20260925 02:43:36.498359 22120 raft_consensus.cc:374] T dc5e802aa3034eab8836b5934f869fba P 935a4168b81f4455b6bcf9ba18b1bf80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:43:36.498389 22120 raft_consensus.cc:729] T dc5e802aa3034eab8836b5934f869fba P 935a4168b81f4455b6bcf9ba18b1bf80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 935a4168b81f4455b6bcf9ba18b1bf80, State: Initialized, Role: FOLLOWER
I20260925 02:43:36.498490 22120 consensus_queue.cc:260] T dc5e802aa3034eab8836b5934f869fba P 935a4168b81f4455b6bcf9ba18b1bf80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77e5b44255714c0db6e1fa4e7130c90f" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 40507 } } peers { permanent_uuid: "eb7b2024cae043fd8827ea003fbbcdbf" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33705 } } peers { permanent_uuid: "935a4168b81f4455b6bcf9ba18b1bf80" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 41903 } }
I20260925 02:43:36.498730 22120 ts_tablet_manager.cc:1428] T dc5e802aa3034eab8836b5934f869fba P 935a4168b81f4455b6bcf9ba18b1bf80: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
W20260925 02:43:36.564538 22091 tablet.cc:2369] T dc5e802aa3034eab8836b5934f869fba P 935a4168b81f4455b6bcf9ba18b1bf80: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 02:43:36.572268 22124 raft_consensus.cc:482] T dc5e802aa3034eab8836b5934f869fba P 77e5b44255714c0db6e1fa4e7130c90f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 02:43:36.572402 22124 raft_consensus.cc:504] T dc5e802aa3034eab8836b5934f869fba P 77e5b44255714c0db6e1fa4e7130c90f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77e5b44255714c0db6e1fa4e7130c90f" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 40507 } } peers { permanent_uuid: "eb7b2024cae043fd8827ea003fbbcdbf" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33705 } } peers { permanent_uuid: "935a4168b81f4455b6bcf9ba18b1bf80" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 41903 } }
I20260925 02:43:36.573486 22124 leader_election.cc:288] T dc5e802aa3034eab8836b5934f869fba P 77e5b44255714c0db6e1fa4e7130c90f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb7b2024cae043fd8827ea003fbbcdbf (127.13.157.130:33705), 935a4168b81f4455b6bcf9ba18b1bf80 (127.13.157.131:41903)
I20260925 02:43:36.577745 22148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:36.573748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.131:41903 (local address 127.13.157.129:53837)
0925 02:43:36.574097 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:36.574109 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:36.574205 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:36.574572 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:36.574575 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:36.574595 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:43:36.574743 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:36.574750 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:36.575768 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:36.575771 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:36.576631 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:36.576641 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:36.577302 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:36.577306 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:36.577409 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:36.577435 (+ 26us) client_negotiation.cc:769] Sending connection context
0925 02:43:36.577507 (+ 72us) client_negotiation.cc:240] Negotiation successful
0925 02:43:36.577577 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"mutex_wait_us":24,"thread_start_us":97,"threads_started":1}
I20260925 02:43:36.577745 22117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:36.574145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.129:53837 (local address 127.13.157.131:41903)
0925 02:43:36.574261 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:36.574265 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:36.574280 (+ 15us) server_negotiation.cc:407] Connection header received
0925 02:43:36.574333 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:36.574336 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:36.574383 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:36.574459 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:43:36.574867 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:36.575629 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:36.576745 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:36.577162 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:36.577276 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:36.577330 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:36.577527 (+ 197us) server_negotiation.cc:299] Negotiation successful
0925 02:43:36.577602 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260925 02:43:36.578235 22126 raft_consensus.cc:482] T dc5e802aa3034eab8836b5934f869fba P eb7b2024cae043fd8827ea003fbbcdbf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 02:43:36.578269 22043 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc5e802aa3034eab8836b5934f869fba" candidate_uuid: "77e5b44255714c0db6e1fa4e7130c90f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "935a4168b81f4455b6bcf9ba18b1bf80" is_pre_election: true
I20260925 02:43:36.578343 22126 raft_consensus.cc:504] T dc5e802aa3034eab8836b5934f869fba P eb7b2024cae043fd8827ea003fbbcdbf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77e5b44255714c0db6e1fa4e7130c90f" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 40507 } } peers { permanent_uuid: "eb7b2024cae043fd8827ea003fbbcdbf" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33705 } } peers { permanent_uuid: "935a4168b81f4455b6bcf9ba18b1bf80" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 41903 } }
I20260925 02:43:36.578516 22043 raft_consensus.cc:2454] T dc5e802aa3034eab8836b5934f869fba P 935a4168b81f4455b6bcf9ba18b1bf80 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 77e5b44255714c0db6e1fa4e7130c90f in term 0.
I20260925 02:43:36.578708 22116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:36.573852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.129:51997 (local address 127.13.157.130:33705)
0925 02:43:36.573978 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:36.573982 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:36.574165 (+ 183us) server_negotiation.cc:407] Connection header received
0925 02:43:36.574338 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:36.574342 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:36.574389 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:36.574451 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:43:36.574707 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:36.575524 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:36.577103 (+ 1579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:36.577605 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:36.577720 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:36.577765 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:36.578526 (+ 761us) server_negotiation.cc:299] Negotiation successful
0925 02:43:36.578604 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260925 02:43:36.579182 22147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:36.573731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.130:33705 (local address 127.13.157.129:51997)
0925 02:43:36.574082 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:36.574096 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:36.574217 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:36.574473 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:36.574476 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:36.574492 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:43:36.574611 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:36.574617 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:36.575639 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:36.575643 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:36.576964 (+ 1321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:36.576976 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:36.578111 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:36.578115 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:36.578222 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:36.578245 (+ 23us) client_negotiation.cc:769] Sending connection context
0925 02:43:36.578962 (+ 717us) client_negotiation.cc:240] Negotiation successful
0925 02:43:36.579049 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"mutex_wait_us":2,"thread_start_us":87,"threads_started":1}
I20260925 02:43:36.579382 22126 leader_election.cc:288] T dc5e802aa3034eab8836b5934f869fba P eb7b2024cae043fd8827ea003fbbcdbf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 77e5b44255714c0db6e1fa4e7130c90f (127.13.157.129:40507), 935a4168b81f4455b6bcf9ba18b1bf80 (127.13.157.131:41903)
I20260925 02:43:36.579799 21902 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc5e802aa3034eab8836b5934f869fba" candidate_uuid: "77e5b44255714c0db6e1fa4e7130c90f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb7b2024cae043fd8827ea003fbbcdbf" is_pre_election: true
I20260925 02:43:36.579926 21902 raft_consensus.cc:2454] T dc5e802aa3034eab8836b5934f869fba P eb7b2024cae043fd8827ea003fbbcdbf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 77e5b44255714c0db6e1fa4e7130c90f in term 0.
I20260925 02:43:36.580052 21691 leader_election.cc:302] T dc5e802aa3034eab8836b5934f869fba P 77e5b44255714c0db6e1fa4e7130c90f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 77e5b44255714c0db6e1fa4e7130c90f, 935a4168b81f4455b6bcf9ba18b1bf80; no voters:
I20260925 02:43:36.580365 22124 raft_consensus.cc:2781] T dc5e802aa3034eab8836b5934f869fba P 77e5b44255714c0db6e1fa4e7130c90f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 02:43:36.580425 22124 raft_consensus.cc:482] T dc5e802aa3034eab8836b5934f869fba P 77e5b44255714c0db6e1fa4e7130c90f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 02:43:36.580464 22124 raft_consensus.cc:3037] T dc5e802aa3034eab8836b5934f869fba P 77e5b44255714c0db6e1fa4e7130c90f [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:43:36.581718 22124 raft_consensus.cc:504] T dc5e802aa3034eab8836b5934f869fba P 77e5b44255714c0db6e1fa4e7130c90f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77e5b44255714c0db6e1fa4e7130c90f" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 40507 } } peers { permanent_uuid: "eb7b2024cae043fd8827ea003fbbcdbf" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33705 } } peers { permanent_uuid: "935a4168b81f4455b6bcf9ba18b1bf80" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 41903 } }
I20260925 02:43:36.582068 22124 leader_election.cc:288] T dc5e802aa3034eab8836b5934f869fba P 77e5b44255714c0db6e1fa4e7130c90f [CANDIDATE]: Term 1 election: Requested vote from peers eb7b2024cae043fd8827ea003fbbcdbf (127.13.157.130:33705), 935a4168b81f4455b6bcf9ba18b1bf80 (127.13.157.131:41903)
I20260925 02:43:36.582589 22043 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc5e802aa3034eab8836b5934f869fba" candidate_uuid: "77e5b44255714c0db6e1fa4e7130c90f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "935a4168b81f4455b6bcf9ba18b1bf80"
I20260925 02:43:36.582698 22043 raft_consensus.cc:3037] T dc5e802aa3034eab8836b5934f869fba P 935a4168b81f4455b6bcf9ba18b1bf80 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:43:36.583271 21902 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc5e802aa3034eab8836b5934f869fba" candidate_uuid: "77e5b44255714c0db6e1fa4e7130c90f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb7b2024cae043fd8827ea003fbbcdbf"
I20260925 02:43:36.583369 21902 raft_consensus.cc:3037] T dc5e802aa3034eab8836b5934f869fba P eb7b2024cae043fd8827ea003fbbcdbf [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:43:36.584237 22117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:36.580194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.130:44375 (local address 127.13.157.131:41903)
0925 02:43:36.580302 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:36.580306 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:36.580493 (+ 187us) server_negotiation.cc:407] Connection header received
0925 02:43:36.580659 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:36.580662 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:36.580702 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:36.580769 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:43:36.581044 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:36.581683 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:36.583106 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:36.583626 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:36.583709 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:36.583749 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:36.584068 (+ 319us) server_negotiation.cc:299] Negotiation successful
0925 02:43:36.584132 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260925 02:43:36.584506 21841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:36.579551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.129:40507 (local address 127.13.157.130:51611)
0925 02:43:36.579667 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:36.579693 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:36.579761 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:36.580693 (+ 932us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:36.580696 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:36.580714 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:43:36.580811 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:36.580817 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:36.581751 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:36.581754 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:36.582960 (+ 1206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:36.582971 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:36.584120 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:36.584124 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:36.584233 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:36.584256 (+ 23us) client_negotiation.cc:769] Sending connection context
0925 02:43:36.584328 (+ 72us) client_negotiation.cc:240] Negotiation successful
0925 02:43:36.584395 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260925 02:43:36.584503 21902 raft_consensus.cc:2454] T dc5e802aa3034eab8836b5934f869fba P eb7b2024cae043fd8827ea003fbbcdbf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 77e5b44255714c0db6e1fa4e7130c90f in term 1.
I20260925 02:43:36.584793 22149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:36.579839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.131:41903 (local address 127.13.157.130:44375)
0925 02:43:36.580410 (+ 571us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:36.580423 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:36.580568 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:36.580797 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:36.580800 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:36.580817 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:43:36.580926 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:36.580932 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:36.581823 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:36.581825 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:36.582971 (+ 1146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:36.582983 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:36.583855 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:36.583858 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:36.583944 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:36.583966 (+ 22us) client_negotiation.cc:769] Sending connection context
0925 02:43:36.584399 (+ 433us) client_negotiation.cc:240] Negotiation successful
0925 02:43:36.584473 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":512,"mutex_wait_us":1,"thread_start_us":64,"threads_started":1}
I20260925 02:43:36.584928 22113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:36.580302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.130:51611 (local address 127.13.157.129:40507)
0925 02:43:36.580396 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:36.580400 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:36.580412 (+ 12us) server_negotiation.cc:407] Connection header received
0925 02:43:36.580445 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:36.580448 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:36.580488 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:36.580597 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:43:36.580899 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:36.581619 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:36.583345 (+ 1726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:36.583885 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:36.583998 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:36.584055 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:36.584753 (+ 698us) server_negotiation.cc:299] Negotiation successful
0925 02:43:36.584824 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260925 02:43:36.585176 22042 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc5e802aa3034eab8836b5934f869fba" candidate_uuid: "eb7b2024cae043fd8827ea003fbbcdbf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "935a4168b81f4455b6bcf9ba18b1bf80" is_pre_election: true
I20260925 02:43:36.585297 21693 leader_election.cc:302] T dc5e802aa3034eab8836b5934f869fba P 77e5b44255714c0db6e1fa4e7130c90f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 77e5b44255714c0db6e1fa4e7130c90f, eb7b2024cae043fd8827ea003fbbcdbf; no voters:
I20260925 02:43:36.585832 21761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc5e802aa3034eab8836b5934f869fba" candidate_uuid: "eb7b2024cae043fd8827ea003fbbcdbf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "77e5b44255714c0db6e1fa4e7130c90f" is_pre_election: true
I20260925 02:43:36.585968 21761 raft_consensus.cc:2379] T dc5e802aa3034eab8836b5934f869fba P 77e5b44255714c0db6e1fa4e7130c90f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb7b2024cae043fd8827ea003fbbcdbf in current term 1: Already voted for candidate 77e5b44255714c0db6e1fa4e7130c90f in this term.
I20260925 02:43:36.586297 21834 leader_election.cc:302] T dc5e802aa3034eab8836b5934f869fba P eb7b2024cae043fd8827ea003fbbcdbf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb7b2024cae043fd8827ea003fbbcdbf; no voters: 77e5b44255714c0db6e1fa4e7130c90f, 935a4168b81f4455b6bcf9ba18b1bf80
I20260925 02:43:36.586544 22126 raft_consensus.cc:2726] T dc5e802aa3034eab8836b5934f869fba P eb7b2024cae043fd8827ea003fbbcdbf [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 02:43:36.587075 22043 raft_consensus.cc:2454] T dc5e802aa3034eab8836b5934f869fba P 935a4168b81f4455b6bcf9ba18b1bf80 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 77e5b44255714c0db6e1fa4e7130c90f in term 1.
I20260925 02:43:36.587251 22124 raft_consensus.cc:2781] T dc5e802aa3034eab8836b5934f869fba P 77e5b44255714c0db6e1fa4e7130c90f [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:43:36.587456 22124 raft_consensus.cc:686] T dc5e802aa3034eab8836b5934f869fba P 77e5b44255714c0db6e1fa4e7130c90f [term 1 LEADER]: Becoming Leader. State: Replica: 77e5b44255714c0db6e1fa4e7130c90f, State: Running, Role: LEADER
I20260925 02:43:36.587699 22124 consensus_queue.cc:237] T dc5e802aa3034eab8836b5934f869fba P 77e5b44255714c0db6e1fa4e7130c90f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77e5b44255714c0db6e1fa4e7130c90f" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 40507 } } peers { permanent_uuid: "eb7b2024cae043fd8827ea003fbbcdbf" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33705 } } peers { permanent_uuid: "935a4168b81f4455b6bcf9ba18b1bf80" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 41903 } }
I20260925 02:43:36.589979 21599 catalog_manager.cc:5495] T dc5e802aa3034eab8836b5934f869fba P 77e5b44255714c0db6e1fa4e7130c90f reported cstate change: term changed from 0 to 1, leader changed from <none> to 77e5b44255714c0db6e1fa4e7130c90f (127.13.157.129). New cstate: current_term: 1 leader_uuid: "77e5b44255714c0db6e1fa4e7130c90f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77e5b44255714c0db6e1fa4e7130c90f" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 40507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb7b2024cae043fd8827ea003fbbcdbf" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33705 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "935a4168b81f4455b6bcf9ba18b1bf80" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 41903 } health_report { overall_health: UNKNOWN } } }
W20260925 02:43:36.601850 21949 tablet.cc:2369] T dc5e802aa3034eab8836b5934f869fba P eb7b2024cae043fd8827ea003fbbcdbf: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 25 02:43:36 dist-test-slave-kznw krb5kdc[21556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304215, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.157.190@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM
I20260925 02:43:36.636528 22157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:36.628486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:43945 (local address 127.0.0.1:48212)
0925 02:43:36.628745 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:36.628759 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:36.628860 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:36.629194 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:36.629197 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:36.629572 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:36.629673 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:36.629678 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:36.630603 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:36.630605 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:36.631227 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:36.631236 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:36.631771 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:36.631774 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:36.631830 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:36.632351 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:36.632363 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:36.633712 (+ 1349us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:43:36.635165 (+ 1453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:36.635173 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:36.635178 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:36.635423 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:36.635642 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:36.635644 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:36.635645 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:36.635714 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:36.636058 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:43:36.636063 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:43:36.636163 (+ 100us) client_negotiation.cc:769] Sending connection context
0925 02:43:36.636251 (+ 88us) client_negotiation.cc:240] Negotiation successful
0925 02:43:36.636363 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":108,"threads_started":1}
I20260925 02:43:36.636538 21651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:36.628570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48212 (local address 127.13.157.190:43945)
0925 02:43:36.628710 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:36.628715 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:36.628828 (+ 113us) server_negotiation.cc:407] Connection header received
0925 02:43:36.628935 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:36.628938 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:36.628987 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:36.629079 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:36.629788 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:36.630501 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:36.631328 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:36.631673 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:36.631776 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:36.633830 (+ 2054us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:36.633833 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:36.633835 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:43:36.633857 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:36.635079 (+ 1222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:36.635501 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:36.635504 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:36.635505 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:36.635575 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:36.635771 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:36.635773 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:36.635774 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:36.635922 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:36.636030 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:36.636292 (+ 262us) server_negotiation.cc:299] Negotiation successful
0925 02:43:36.636416 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":29,"server-negotiator.queue_time_us":52}
I20260925 02:43:36.640112 21599 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.13.157.190@KRBTEST.COM'} at 127.0.0.1:48212:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260925 02:43:36.640255 21808 tablet.cc:2369] T dc5e802aa3034eab8836b5934f869fba P 77e5b44255714c0db6e1fa4e7130c90f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 02:43:36.640898 21599 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 02:43:36.645661 21740 tablet_service.cc:1467] Processing CreateTablet for tablet d0d33d5d6b604147ae09946e1a3a48c7 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=9aae293fd0d84d4ba01ecab1c33b251b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 02:43:36.645709 21882 tablet_service.cc:1467] Processing CreateTablet for tablet d0d33d5d6b604147ae09946e1a3a48c7 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=9aae293fd0d84d4ba01ecab1c33b251b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 02:43:36.646083 21740 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d0d33d5d6b604147ae09946e1a3a48c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:43:36.646136 21882 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d0d33d5d6b604147ae09946e1a3a48c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:43:36.648455 22118 tablet_bootstrap.cc:492] T d0d33d5d6b604147ae09946e1a3a48c7 P 77e5b44255714c0db6e1fa4e7130c90f: Bootstrap starting.
I20260925 02:43:36.648541 22119 tablet_bootstrap.cc:492] T d0d33d5d6b604147ae09946e1a3a48c7 P eb7b2024cae043fd8827ea003fbbcdbf: Bootstrap starting.
I20260925 02:43:36.648772 22023 tablet_service.cc:1467] Processing CreateTablet for tablet d0d33d5d6b604147ae09946e1a3a48c7 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=9aae293fd0d84d4ba01ecab1c33b251b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 02:43:36.649132 22023 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d0d33d5d6b604147ae09946e1a3a48c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:43:36.649411 22118 tablet_bootstrap.cc:654] T d0d33d5d6b604147ae09946e1a3a48c7 P 77e5b44255714c0db6e1fa4e7130c90f: Neither blocks nor log segments found. Creating new log.
I20260925 02:43:36.649519 22119 tablet_bootstrap.cc:654] T d0d33d5d6b604147ae09946e1a3a48c7 P eb7b2024cae043fd8827ea003fbbcdbf: Neither blocks nor log segments found. Creating new log.
I20260925 02:43:36.650701 22118 tablet_bootstrap.cc:492] T d0d33d5d6b604147ae09946e1a3a48c7 P 77e5b44255714c0db6e1fa4e7130c90f: No bootstrap required, opened a new log
I20260925 02:43:36.650701 22119 tablet_bootstrap.cc:492] T d0d33d5d6b604147ae09946e1a3a48c7 P eb7b2024cae043fd8827ea003fbbcdbf: No bootstrap required, opened a new log
I20260925 02:43:36.650804 22118 ts_tablet_manager.cc:1397] T d0d33d5d6b604147ae09946e1a3a48c7 P 77e5b44255714c0db6e1fa4e7130c90f: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260925 02:43:36.650804 22119 ts_tablet_manager.cc:1397] T d0d33d5d6b604147ae09946e1a3a48c7 P eb7b2024cae043fd8827ea003fbbcdbf: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260925 02:43:36.651295 22119 raft_consensus.cc:348] T d0d33d5d6b604147ae09946e1a3a48c7 P eb7b2024cae043fd8827ea003fbbcdbf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77e5b44255714c0db6e1fa4e7130c90f" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 40507 } } peers { permanent_uuid: "eb7b2024cae043fd8827ea003fbbcdbf" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33705 } } peers { permanent_uuid: "935a4168b81f4455b6bcf9ba18b1bf80" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 41903 } }
I20260925 02:43:36.651295 22118 raft_consensus.cc:348] T d0d33d5d6b604147ae09946e1a3a48c7 P 77e5b44255714c0db6e1fa4e7130c90f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77e5b44255714c0db6e1fa4e7130c90f" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 40507 } } peers { permanent_uuid: "eb7b2024cae043fd8827ea003fbbcdbf" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33705 } } peers { permanent_uuid: "935a4168b81f4455b6bcf9ba18b1bf80" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 41903 } }
I20260925 02:43:36.651419 22119 raft_consensus.cc:374] T d0d33d5d6b604147ae09946e1a3a48c7 P eb7b2024cae043fd8827ea003fbbcdbf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:43:36.651419 22118 raft_consensus.cc:374] T d0d33d5d6b604147ae09946e1a3a48c7 P 77e5b44255714c0db6e1fa4e7130c90f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:43:36.651458 22119 raft_consensus.cc:729] T d0d33d5d6b604147ae09946e1a3a48c7 P eb7b2024cae043fd8827ea003fbbcdbf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb7b2024cae043fd8827ea003fbbcdbf, State: Initialized, Role: FOLLOWER
I20260925 02:43:36.651465 22118 raft_consensus.cc:729] T d0d33d5d6b604147ae09946e1a3a48c7 P 77e5b44255714c0db6e1fa4e7130c90f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77e5b44255714c0db6e1fa4e7130c90f, State: Initialized, Role: FOLLOWER
I20260925 02:43:36.651625 22120 tablet_bootstrap.cc:492] T d0d33d5d6b604147ae09946e1a3a48c7 P 935a4168b81f4455b6bcf9ba18b1bf80: Bootstrap starting.
I20260925 02:43:36.651594 22118 consensus_queue.cc:260] T d0d33d5d6b604147ae09946e1a3a48c7 P 77e5b44255714c0db6e1fa4e7130c90f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77e5b44255714c0db6e1fa4e7130c90f" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 40507 } } peers { permanent_uuid: "eb7b2024cae043fd8827ea003fbbcdbf" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33705 } } peers { permanent_uuid: "935a4168b81f4455b6bcf9ba18b1bf80" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 41903 } }
I20260925 02:43:36.651607 22119 consensus_queue.cc:260] T d0d33d5d6b604147ae09946e1a3a48c7 P eb7b2024cae043fd8827ea003fbbcdbf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77e5b44255714c0db6e1fa4e7130c90f" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 40507 } } peers { permanent_uuid: "eb7b2024cae043fd8827ea003fbbcdbf" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33705 } } peers { permanent_uuid: "935a4168b81f4455b6bcf9ba18b1bf80" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 41903 } }
I20260925 02:43:36.651878 22119 ts_tablet_manager.cc:1428] T d0d33d5d6b604147ae09946e1a3a48c7 P eb7b2024cae043fd8827ea003fbbcdbf: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260925 02:43:36.651878 22118 ts_tablet_manager.cc:1428] T d0d33d5d6b604147ae09946e1a3a48c7 P 77e5b44255714c0db6e1fa4e7130c90f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260925 02:43:36.651855 22126 tablet_replica.cc:439] T d0d33d5d6b604147ae09946e1a3a48c7 P eb7b2024cae043fd8827ea003fbbcdbf: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77e5b44255714c0db6e1fa4e7130c90f" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 40507 } } peers { permanent_uuid: "eb7b2024cae043fd8827ea003fbbcdbf" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33705 } } peers { permanent_uuid: "935a4168b81f4455b6bcf9ba18b1bf80" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 41903 } } }
I20260925 02:43:36.652004 22126 tablet_replica.cc:442] T d0d33d5d6b604147ae09946e1a3a48c7 P eb7b2024cae043fd8827ea003fbbcdbf: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 02:43:36.652416 22120 tablet_bootstrap.cc:654] T d0d33d5d6b604147ae09946e1a3a48c7 P 935a4168b81f4455b6bcf9ba18b1bf80: Neither blocks nor log segments found. Creating new log.
I20260925 02:43:36.652920 22150 tablet_replica.cc:439] T d0d33d5d6b604147ae09946e1a3a48c7 P 77e5b44255714c0db6e1fa4e7130c90f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77e5b44255714c0db6e1fa4e7130c90f" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 40507 } } peers { permanent_uuid: "eb7b2024cae043fd8827ea003fbbcdbf" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33705 } } peers { permanent_uuid: "935a4168b81f4455b6bcf9ba18b1bf80" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 41903 } } }
I20260925 02:43:36.653046 22150 tablet_replica.cc:442] T d0d33d5d6b604147ae09946e1a3a48c7 P 77e5b44255714c0db6e1fa4e7130c90f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 02:43:36.653535 22120 tablet_bootstrap.cc:492] T d0d33d5d6b604147ae09946e1a3a48c7 P 935a4168b81f4455b6bcf9ba18b1bf80: No bootstrap required, opened a new log
I20260925 02:43:36.653614 22120 ts_tablet_manager.cc:1397] T d0d33d5d6b604147ae09946e1a3a48c7 P 935a4168b81f4455b6bcf9ba18b1bf80: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260925 02:43:36.653990 22120 raft_consensus.cc:348] T d0d33d5d6b604147ae09946e1a3a48c7 P 935a4168b81f4455b6bcf9ba18b1bf80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77e5b44255714c0db6e1fa4e7130c90f" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 40507 } } peers { permanent_uuid: "eb7b2024cae043fd8827ea003fbbcdbf" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33705 } } peers { permanent_uuid: "935a4168b81f4455b6bcf9ba18b1bf80" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 41903 } }
I20260925 02:43:36.654090 22120 raft_consensus.cc:374] T d0d33d5d6b604147ae09946e1a3a48c7 P 935a4168b81f4455b6bcf9ba18b1bf80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:43:36.654124 22120 raft_consensus.cc:729] T d0d33d5d6b604147ae09946e1a3a48c7 P 935a4168b81f4455b6bcf9ba18b1bf80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 935a4168b81f4455b6bcf9ba18b1bf80, State: Initialized, Role: FOLLOWER
I20260925 02:43:36.654239 22120 consensus_queue.cc:260] T d0d33d5d6b604147ae09946e1a3a48c7 P 935a4168b81f4455b6bcf9ba18b1bf80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77e5b44255714c0db6e1fa4e7130c90f" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 40507 } } peers { permanent_uuid: "eb7b2024cae043fd8827ea003fbbcdbf" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33705 } } peers { permanent_uuid: "935a4168b81f4455b6bcf9ba18b1bf80" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 41903 } }
I20260925 02:43:36.654484 22120 ts_tablet_manager.cc:1428] T d0d33d5d6b604147ae09946e1a3a48c7 P 935a4168b81f4455b6bcf9ba18b1bf80: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260925 02:43:36.654668 22128 tablet_replica.cc:439] T d0d33d5d6b604147ae09946e1a3a48c7 P 935a4168b81f4455b6bcf9ba18b1bf80: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77e5b44255714c0db6e1fa4e7130c90f" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 40507 } } peers { permanent_uuid: "eb7b2024cae043fd8827ea003fbbcdbf" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33705 } } peers { permanent_uuid: "935a4168b81f4455b6bcf9ba18b1bf80" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 41903 } } }
I20260925 02:43:36.654805 22128 tablet_replica.cc:442] T d0d33d5d6b604147ae09946e1a3a48c7 P 935a4168b81f4455b6bcf9ba18b1bf80: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 02:43:36.940408 22128 raft_consensus.cc:482] T d0d33d5d6b604147ae09946e1a3a48c7 P 935a4168b81f4455b6bcf9ba18b1bf80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 02:43:36.940553 22128 raft_consensus.cc:504] T d0d33d5d6b604147ae09946e1a3a48c7 P 935a4168b81f4455b6bcf9ba18b1bf80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77e5b44255714c0db6e1fa4e7130c90f" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 40507 } } peers { permanent_uuid: "eb7b2024cae043fd8827ea003fbbcdbf" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33705 } } peers { permanent_uuid: "935a4168b81f4455b6bcf9ba18b1bf80" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 41903 } }
I20260925 02:43:36.940963 22128 leader_election.cc:288] T d0d33d5d6b604147ae09946e1a3a48c7 P 935a4168b81f4455b6bcf9ba18b1bf80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 77e5b44255714c0db6e1fa4e7130c90f (127.13.157.129:40507), eb7b2024cae043fd8827ea003fbbcdbf (127.13.157.130:33705)
I20260925 02:43:36.941406 21902 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0d33d5d6b604147ae09946e1a3a48c7" candidate_uuid: "935a4168b81f4455b6bcf9ba18b1bf80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb7b2024cae043fd8827ea003fbbcdbf" is_pre_election: true
I20260925 02:43:36.941421 21761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0d33d5d6b604147ae09946e1a3a48c7" candidate_uuid: "935a4168b81f4455b6bcf9ba18b1bf80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "77e5b44255714c0db6e1fa4e7130c90f" is_pre_election: true
I20260925 02:43:36.941548 21902 raft_consensus.cc:2454] T d0d33d5d6b604147ae09946e1a3a48c7 P eb7b2024cae043fd8827ea003fbbcdbf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 935a4168b81f4455b6bcf9ba18b1bf80 in term 0.
I20260925 02:43:36.941593 21761 raft_consensus.cc:2454] T d0d33d5d6b604147ae09946e1a3a48c7 P 77e5b44255714c0db6e1fa4e7130c90f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 935a4168b81f4455b6bcf9ba18b1bf80 in term 0.
I20260925 02:43:36.941841 21976 leader_election.cc:302] T d0d33d5d6b604147ae09946e1a3a48c7 P 935a4168b81f4455b6bcf9ba18b1bf80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 935a4168b81f4455b6bcf9ba18b1bf80, eb7b2024cae043fd8827ea003fbbcdbf; no voters:
I20260925 02:43:36.942026 22128 raft_consensus.cc:2781] T d0d33d5d6b604147ae09946e1a3a48c7 P 935a4168b81f4455b6bcf9ba18b1bf80 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 02:43:36.942090 22128 raft_consensus.cc:482] T d0d33d5d6b604147ae09946e1a3a48c7 P 935a4168b81f4455b6bcf9ba18b1bf80 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 02:43:36.942134 22128 raft_consensus.cc:3037] T d0d33d5d6b604147ae09946e1a3a48c7 P 935a4168b81f4455b6bcf9ba18b1bf80 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:43:36.943176 22128 raft_consensus.cc:504] T d0d33d5d6b604147ae09946e1a3a48c7 P 935a4168b81f4455b6bcf9ba18b1bf80 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77e5b44255714c0db6e1fa4e7130c90f" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 40507 } } peers { permanent_uuid: "eb7b2024cae043fd8827ea003fbbcdbf" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33705 } } peers { permanent_uuid: "935a4168b81f4455b6bcf9ba18b1bf80" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 41903 } }
I20260925 02:43:36.943462 22128 leader_election.cc:288] T d0d33d5d6b604147ae09946e1a3a48c7 P 935a4168b81f4455b6bcf9ba18b1bf80 [CANDIDATE]: Term 1 election: Requested vote from peers 77e5b44255714c0db6e1fa4e7130c90f (127.13.157.129:40507), eb7b2024cae043fd8827ea003fbbcdbf (127.13.157.130:33705)
I20260925 02:43:36.943774 21902 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0d33d5d6b604147ae09946e1a3a48c7" candidate_uuid: "935a4168b81f4455b6bcf9ba18b1bf80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb7b2024cae043fd8827ea003fbbcdbf"
I20260925 02:43:36.943784 21761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0d33d5d6b604147ae09946e1a3a48c7" candidate_uuid: "935a4168b81f4455b6bcf9ba18b1bf80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "77e5b44255714c0db6e1fa4e7130c90f"
I20260925 02:43:36.943887 21902 raft_consensus.cc:3037] T d0d33d5d6b604147ae09946e1a3a48c7 P eb7b2024cae043fd8827ea003fbbcdbf [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:43:36.943895 21761 raft_consensus.cc:3037] T d0d33d5d6b604147ae09946e1a3a48c7 P 77e5b44255714c0db6e1fa4e7130c90f [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:43:36.944773 21902 raft_consensus.cc:2454] T d0d33d5d6b604147ae09946e1a3a48c7 P eb7b2024cae043fd8827ea003fbbcdbf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 935a4168b81f4455b6bcf9ba18b1bf80 in term 1.
I20260925 02:43:36.944868 21761 raft_consensus.cc:2454] T d0d33d5d6b604147ae09946e1a3a48c7 P 77e5b44255714c0db6e1fa4e7130c90f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 935a4168b81f4455b6bcf9ba18b1bf80 in term 1.
I20260925 02:43:36.945075 21976 leader_election.cc:302] T d0d33d5d6b604147ae09946e1a3a48c7 P 935a4168b81f4455b6bcf9ba18b1bf80 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 935a4168b81f4455b6bcf9ba18b1bf80, eb7b2024cae043fd8827ea003fbbcdbf; no voters:
I20260925 02:43:36.945221 22128 raft_consensus.cc:2781] T d0d33d5d6b604147ae09946e1a3a48c7 P 935a4168b81f4455b6bcf9ba18b1bf80 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:43:36.945310 22128 raft_consensus.cc:686] T d0d33d5d6b604147ae09946e1a3a48c7 P 935a4168b81f4455b6bcf9ba18b1bf80 [term 1 LEADER]: Becoming Leader. State: Replica: 935a4168b81f4455b6bcf9ba18b1bf80, State: Running, Role: LEADER
I20260925 02:43:36.945467 22128 consensus_queue.cc:237] T d0d33d5d6b604147ae09946e1a3a48c7 P 935a4168b81f4455b6bcf9ba18b1bf80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77e5b44255714c0db6e1fa4e7130c90f" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 40507 } } peers { permanent_uuid: "eb7b2024cae043fd8827ea003fbbcdbf" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33705 } } peers { permanent_uuid: "935a4168b81f4455b6bcf9ba18b1bf80" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 41903 } }
I20260925 02:43:36.946120 22125 tablet_replica.cc:439] T d0d33d5d6b604147ae09946e1a3a48c7 P 935a4168b81f4455b6bcf9ba18b1bf80: TxnStatusTablet state changed. Reason: New leader 935a4168b81f4455b6bcf9ba18b1bf80. Latest consensus state: current_term: 1 leader_uuid: "935a4168b81f4455b6bcf9ba18b1bf80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77e5b44255714c0db6e1fa4e7130c90f" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 40507 } } peers { permanent_uuid: "eb7b2024cae043fd8827ea003fbbcdbf" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33705 } } peers { permanent_uuid: "935a4168b81f4455b6bcf9ba18b1bf80" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 41903 } } }
I20260925 02:43:36.946228 22125 tablet_replica.cc:442] T d0d33d5d6b604147ae09946e1a3a48c7 P 935a4168b81f4455b6bcf9ba18b1bf80: This TxnStatusTablet replica's current role is: LEADER
I20260925 02:43:36.946604 22162 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260925 02:43:36.946768 22162 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260925 02:43:36.947201 21601 catalog_manager.cc:5495] T d0d33d5d6b604147ae09946e1a3a48c7 P 935a4168b81f4455b6bcf9ba18b1bf80 reported cstate change: term changed from 0 to 1, leader changed from <none> to 935a4168b81f4455b6bcf9ba18b1bf80 (127.13.157.131). New cstate: current_term: 1 leader_uuid: "935a4168b81f4455b6bcf9ba18b1bf80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77e5b44255714c0db6e1fa4e7130c90f" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 40507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb7b2024cae043fd8827ea003fbbcdbf" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33705 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "935a4168b81f4455b6bcf9ba18b1bf80" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 41903 } health_report { overall_health: HEALTHY } } }
I20260925 02:43:37.025149 22117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:37.020923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43302 (local address 127.13.157.131:41903)
0925 02:43:37.021046 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:37.021049 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:37.021064 (+ 15us) server_negotiation.cc:407] Connection header received
0925 02:43:37.021259 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:37.021263 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:37.021320 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:37.021383 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 02:43:37.021697 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:37.022470 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:37.023776 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:37.024133 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:37.024232 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:37.024657 (+ 425us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 02:43:37.024752 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 02:43:37.024816 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:37.024940 (+ 124us) server_negotiation.cc:299] Negotiation successful
0925 02:43:37.024998 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260925 02:43:37.025166 22157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:37.020799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.131:41903 (local address 127.0.0.1:43302)
0925 02:43:37.020979 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:37.020994 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:37.021104 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:37.021435 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:37.021439 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:37.021448 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 02:43:37.021581 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:37.021588 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:37.022587 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:37.022591 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:37.023656 (+ 1065us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:37.023668 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:37.024323 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:37.024327 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:37.024428 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:37.024487 (+ 59us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 02:43:37.024828 (+ 341us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 02:43:37.024846 (+ 18us) client_negotiation.cc:769] Sending connection context
0925 02:43:37.024917 (+ 71us) client_negotiation.cc:240] Negotiation successful
0925 02:43:37.025021 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260925 02:43:37.028234 21761 raft_consensus.cc:1261] T d0d33d5d6b604147ae09946e1a3a48c7 P 77e5b44255714c0db6e1fa4e7130c90f [term 1 FOLLOWER]: Refusing update from remote peer 935a4168b81f4455b6bcf9ba18b1bf80: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:43:37.028258 21902 raft_consensus.cc:1261] T d0d33d5d6b604147ae09946e1a3a48c7 P eb7b2024cae043fd8827ea003fbbcdbf [term 1 FOLLOWER]: Refusing update from remote peer 935a4168b81f4455b6bcf9ba18b1bf80: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:43:37.029165 22128 consensus_queue.cc:1035] T d0d33d5d6b604147ae09946e1a3a48c7 P 935a4168b81f4455b6bcf9ba18b1bf80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb7b2024cae043fd8827ea003fbbcdbf" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:43:37.029405 22125 consensus_queue.cc:1035] T d0d33d5d6b604147ae09946e1a3a48c7 P 935a4168b81f4455b6bcf9ba18b1bf80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "77e5b44255714c0db6e1fa4e7130c90f" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 40507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:43:37.031070 22150 tablet_replica.cc:439] T d0d33d5d6b604147ae09946e1a3a48c7 P 77e5b44255714c0db6e1fa4e7130c90f: TxnStatusTablet state changed. Reason: New leader 935a4168b81f4455b6bcf9ba18b1bf80. Latest consensus state: current_term: 1 leader_uuid: "935a4168b81f4455b6bcf9ba18b1bf80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77e5b44255714c0db6e1fa4e7130c90f" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 40507 } } peers { permanent_uuid: "eb7b2024cae043fd8827ea003fbbcdbf" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33705 } } peers { permanent_uuid: "935a4168b81f4455b6bcf9ba18b1bf80" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 41903 } } }
I20260925 02:43:37.031194 22150 tablet_replica.cc:442] T d0d33d5d6b604147ae09946e1a3a48c7 P 77e5b44255714c0db6e1fa4e7130c90f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 02:43:37.031203 22126 tablet_replica.cc:439] T d0d33d5d6b604147ae09946e1a3a48c7 P eb7b2024cae043fd8827ea003fbbcdbf: TxnStatusTablet state changed. Reason: New leader 935a4168b81f4455b6bcf9ba18b1bf80. Latest consensus state: current_term: 1 leader_uuid: "935a4168b81f4455b6bcf9ba18b1bf80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77e5b44255714c0db6e1fa4e7130c90f" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 40507 } } peers { permanent_uuid: "eb7b2024cae043fd8827ea003fbbcdbf" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33705 } } peers { permanent_uuid: "935a4168b81f4455b6bcf9ba18b1bf80" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 41903 } } }
I20260925 02:43:37.031296 22126 tablet_replica.cc:442] T d0d33d5d6b604147ae09946e1a3a48c7 P eb7b2024cae043fd8827ea003fbbcdbf: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 02:43:37.033016 22124 consensus_queue.cc:1035] T dc5e802aa3034eab8836b5934f869fba P 77e5b44255714c0db6e1fa4e7130c90f [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb7b2024cae043fd8827ea003fbbcdbf" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:43:37.036720 22150 consensus_queue.cc:1035] T dc5e802aa3034eab8836b5934f869fba P 77e5b44255714c0db6e1fa4e7130c90f [LEADER]: Connected to new peer: Peer: permanent_uuid: "935a4168b81f4455b6bcf9ba18b1bf80" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 41903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:43:37.040217 22125 tablet_replica.cc:439] T d0d33d5d6b604147ae09946e1a3a48c7 P 935a4168b81f4455b6bcf9ba18b1bf80: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "935a4168b81f4455b6bcf9ba18b1bf80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77e5b44255714c0db6e1fa4e7130c90f" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 40507 } } peers { permanent_uuid: "eb7b2024cae043fd8827ea003fbbcdbf" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33705 } } peers { permanent_uuid: "935a4168b81f4455b6bcf9ba18b1bf80" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 41903 } } }
I20260925 02:43:37.040331 22125 tablet_replica.cc:442] T d0d33d5d6b604147ae09946e1a3a48c7 P 935a4168b81f4455b6bcf9ba18b1bf80: This TxnStatusTablet replica's current role is: LEADER
I20260925 02:43:37.041884 22126 tablet_replica.cc:439] T d0d33d5d6b604147ae09946e1a3a48c7 P eb7b2024cae043fd8827ea003fbbcdbf: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "935a4168b81f4455b6bcf9ba18b1bf80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77e5b44255714c0db6e1fa4e7130c90f" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 40507 } } peers { permanent_uuid: "eb7b2024cae043fd8827ea003fbbcdbf" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33705 } } peers { permanent_uuid: "935a4168b81f4455b6bcf9ba18b1bf80" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 41903 } } }
I20260925 02:43:37.042001 22126 tablet_replica.cc:442] T d0d33d5d6b604147ae09946e1a3a48c7 P eb7b2024cae043fd8827ea003fbbcdbf: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 02:43:37.043195 22128 tablet_replica.cc:439] T d0d33d5d6b604147ae09946e1a3a48c7 P 935a4168b81f4455b6bcf9ba18b1bf80: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "935a4168b81f4455b6bcf9ba18b1bf80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77e5b44255714c0db6e1fa4e7130c90f" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 40507 } } peers { permanent_uuid: "eb7b2024cae043fd8827ea003fbbcdbf" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33705 } } peers { permanent_uuid: "935a4168b81f4455b6bcf9ba18b1bf80" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 41903 } } }
I20260925 02:43:37.043300 22128 tablet_replica.cc:442] T d0d33d5d6b604147ae09946e1a3a48c7 P 935a4168b81f4455b6bcf9ba18b1bf80: This TxnStatusTablet replica's current role is: LEADER
I20260925 02:43:37.044448 22166 mvcc.cc:205] Tried to move back new op lower bound from 7333086072942718976 to 7333086072609988608. Current Snapshot: MvccSnapshot[applied={T|T < 7333086072942718976 or (T in {7333086072942718976})}]
I20260925 02:43:37.045279 22150 tablet_replica.cc:439] T d0d33d5d6b604147ae09946e1a3a48c7 P 77e5b44255714c0db6e1fa4e7130c90f: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "935a4168b81f4455b6bcf9ba18b1bf80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77e5b44255714c0db6e1fa4e7130c90f" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 40507 } } peers { permanent_uuid: "eb7b2024cae043fd8827ea003fbbcdbf" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33705 } } peers { permanent_uuid: "935a4168b81f4455b6bcf9ba18b1bf80" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 41903 } } }
I20260925 02:43:37.045382 22150 tablet_replica.cc:442] T d0d33d5d6b604147ae09946e1a3a48c7 P 77e5b44255714c0db6e1fa4e7130c90f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 02:43:37.050251 22113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:37.045897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35988 (local address 127.13.157.129:40507)
0925 02:43:37.046022 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:37.046026 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:37.046280 (+ 254us) server_negotiation.cc:407] Connection header received
0925 02:43:37.046484 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:37.046487 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:37.046563 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:37.046645 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 02:43:37.047073 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:37.047570 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:37.048793 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:37.049002 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:37.049125 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:37.049416 (+ 291us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 02:43:37.049826 (+ 410us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 02:43:37.049906 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:37.050072 (+ 166us) server_negotiation.cc:299] Negotiation successful
0925 02:43:37.050127 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260925 02:43:37.060879 22117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:37.056995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.129:56863 (local address 127.13.157.131:41903)
0925 02:43:37.057116 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:37.057121 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:37.057349 (+ 228us) server_negotiation.cc:407] Connection header received
0925 02:43:37.057522 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:37.057524 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:37.057570 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:37.057633 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 02:43:37.057945 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:37.058494 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:37.059779 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:37.060002 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:37.060083 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:37.060412 (+ 329us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 02:43:37.060483 (+ 71us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 02:43:37.060536 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:37.060716 (+ 180us) server_negotiation.cc:299] Negotiation successful
0925 02:43:37.060768 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260925 02:43:37.060921 22180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:37.056830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.131:41903 (local address 127.13.157.129:56863)
0925 02:43:37.057257 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:37.057271 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:37.057388 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:37.057674 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:37.057677 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:37.057686 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 02:43:37.057820 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:37.057826 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:37.058616 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:37.058620 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:37.059657 (+ 1037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:37.059669 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:37.060116 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:37.060119 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:37.060220 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:37.060275 (+ 55us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 02:43:37.060567 (+ 292us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 02:43:37.060574 (+ 7us) client_negotiation.cc:769] Sending connection context
0925 02:43:37.060644 (+ 70us) client_negotiation.cc:240] Negotiation successful
0925 02:43:37.060724 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":100,"threads_started":1}
I20260925 02:43:37.087246 22184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:37.083096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.129:40507 (local address 127.13.157.131:53463)
0925 02:43:37.083396 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:37.083409 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:37.083531 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:37.083762 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:37.083765 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:37.083772 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 02:43:37.083874 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:37.083880 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:37.085062 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:37.085065 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:37.085951 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:37.085962 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:37.086545 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:37.086548 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:37.086624 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:37.086674 (+ 50us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 02:43:37.086964 (+ 290us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 02:43:37.086970 (+ 6us) client_negotiation.cc:769] Sending connection context
0925 02:43:37.087035 (+ 65us) client_negotiation.cc:240] Negotiation successful
0925 02:43:37.087122 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":102,"threads_started":1}
I20260925 02:43:37.087244 22113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:37.083161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.131:53463 (local address 127.13.157.129:40507)
0925 02:43:37.083274 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:37.083279 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:37.083474 (+ 195us) server_negotiation.cc:407] Connection header received
0925 02:43:37.083636 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:37.083639 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:37.083687 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:37.083744 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 02:43:37.083993 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:37.084944 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:37.086056 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:37.086431 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:37.086526 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:37.086787 (+ 261us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 02:43:37.086866 (+ 79us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 02:43:37.086962 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:37.087041 (+ 79us) server_negotiation.cc:299] Negotiation successful
0925 02:43:37.087101 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260925 02:43:37.109154 21599 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48204:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 02:43:37.109347 21599 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48204:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 02:43:37.111861 21599 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9120630de98a487eb499476860ce3215: Sending DeleteTablet for 3 replicas of tablet dc5e802aa3034eab8836b5934f869fba
I20260925 02:43:37.112564 21882 tablet_service.cc:1514] Processing DeleteTablet for tablet dc5e802aa3034eab8836b5934f869fba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:43:37 UTC) from {username='oryx', principal='oryx/127.13.157.190@KRBTEST.COM'} at 127.0.0.1:42642
I20260925 02:43:37.112780 22023 tablet_service.cc:1514] Processing DeleteTablet for tablet dc5e802aa3034eab8836b5934f869fba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:43:37 UTC) from {username='oryx', principal='oryx/127.13.157.190@KRBTEST.COM'} at 127.0.0.1:43284
I20260925 02:43:37.112929 22187 tablet_replica.cc:330] T dc5e802aa3034eab8836b5934f869fba P eb7b2024cae043fd8827ea003fbbcdbf: stopping tablet replica
I20260925 02:43:37.113098 22187 raft_consensus.cc:2229] T dc5e802aa3034eab8836b5934f869fba P eb7b2024cae043fd8827ea003fbbcdbf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 02:43:37.113222 22187 raft_consensus.cc:2258] T dc5e802aa3034eab8836b5934f869fba P eb7b2024cae043fd8827ea003fbbcdbf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:43:37.113302 21740 tablet_service.cc:1514] Processing DeleteTablet for tablet dc5e802aa3034eab8836b5934f869fba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:43:37 UTC) from {username='oryx', principal='oryx/127.13.157.190@KRBTEST.COM'} at 127.0.0.1:35980
I20260925 02:43:37.113880 22187 ts_tablet_manager.cc:1905] T dc5e802aa3034eab8836b5934f869fba P eb7b2024cae043fd8827ea003fbbcdbf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:43:37.116684 22187 ts_tablet_manager.cc:1918] T dc5e802aa3034eab8836b5934f869fba P eb7b2024cae043fd8827ea003fbbcdbf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 02:43:37.116760 22187 log.cc:1196] T dc5e802aa3034eab8836b5934f869fba P eb7b2024cae043fd8827ea003fbbcdbf: Deleting WAL directory at /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790304178325258-13942-0/minicluster-data/ts-1/wal/wals/dc5e802aa3034eab8836b5934f869fba
I20260925 02:43:37.116981 22187 ts_tablet_manager.cc:1939] T dc5e802aa3034eab8836b5934f869fba P eb7b2024cae043fd8827ea003fbbcdbf: Deleting consensus metadata
I20260925 02:43:37.117738 21588 catalog_manager.cc:4841] TS eb7b2024cae043fd8827ea003fbbcdbf (127.13.157.130:33705): tablet dc5e802aa3034eab8836b5934f869fba (table test-table [id=ff3a161f2faf427fb8f51126955f0d1f]) successfully deleted
I20260925 02:43:37.119071 22189 tablet_replica.cc:330] T dc5e802aa3034eab8836b5934f869fba P 77e5b44255714c0db6e1fa4e7130c90f: stopping tablet replica
I20260925 02:43:37.119206 22189 raft_consensus.cc:2229] T dc5e802aa3034eab8836b5934f869fba P 77e5b44255714c0db6e1fa4e7130c90f [term 1 LEADER]: Raft consensus shutting down.
I20260925 02:43:37.119549 22189 raft_consensus.cc:2258] T dc5e802aa3034eab8836b5934f869fba P 77e5b44255714c0db6e1fa4e7130c90f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:43:37.119813 22188 tablet_replica.cc:330] T dc5e802aa3034eab8836b5934f869fba P 935a4168b81f4455b6bcf9ba18b1bf80: stopping tablet replica
I20260925 02:43:37.119954 22188 raft_consensus.cc:2229] T dc5e802aa3034eab8836b5934f869fba P 935a4168b81f4455b6bcf9ba18b1bf80 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 02:43:37.120075 22188 raft_consensus.cc:2258] T dc5e802aa3034eab8836b5934f869fba P 935a4168b81f4455b6bcf9ba18b1bf80 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:43:37.120795 22188 ts_tablet_manager.cc:1905] T dc5e802aa3034eab8836b5934f869fba P 935a4168b81f4455b6bcf9ba18b1bf80: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:43:37.121742 22189 ts_tablet_manager.cc:1905] T dc5e802aa3034eab8836b5934f869fba P 77e5b44255714c0db6e1fa4e7130c90f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:43:37.124872 22188 ts_tablet_manager.cc:1918] T dc5e802aa3034eab8836b5934f869fba P 935a4168b81f4455b6bcf9ba18b1bf80: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 02:43:37.124955 22188 log.cc:1196] T dc5e802aa3034eab8836b5934f869fba P 935a4168b81f4455b6bcf9ba18b1bf80: Deleting WAL directory at /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790304178325258-13942-0/minicluster-data/ts-2/wal/wals/dc5e802aa3034eab8836b5934f869fba
I20260925 02:43:37.125216 22188 ts_tablet_manager.cc:1939] T dc5e802aa3034eab8836b5934f869fba P 935a4168b81f4455b6bcf9ba18b1bf80: Deleting consensus metadata
I20260925 02:43:37.125929 21586 catalog_manager.cc:4841] TS 935a4168b81f4455b6bcf9ba18b1bf80 (127.13.157.131:41903): tablet dc5e802aa3034eab8836b5934f869fba (table test-table [id=ff3a161f2faf427fb8f51126955f0d1f]) successfully deleted
Sep 25 02:43:37 dist-test-slave-kznw krb5kdc[21556](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304217, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
I20260925 02:43:37.128314 22189 ts_tablet_manager.cc:1918] T dc5e802aa3034eab8836b5934f869fba P 77e5b44255714c0db6e1fa4e7130c90f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 02:43:37.128393 22189 log.cc:1196] T dc5e802aa3034eab8836b5934f869fba P 77e5b44255714c0db6e1fa4e7130c90f: Deleting WAL directory at /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790304178325258-13942-0/minicluster-data/ts-0/wal/wals/dc5e802aa3034eab8836b5934f869fba
I20260925 02:43:37.128647 22189 ts_tablet_manager.cc:1939] T dc5e802aa3034eab8836b5934f869fba P 77e5b44255714c0db6e1fa4e7130c90f: Deleting consensus metadata
I20260925 02:43:37.129350 21586 catalog_manager.cc:4841] TS 77e5b44255714c0db6e1fa4e7130c90f (127.13.157.129:40507): tablet dc5e802aa3034eab8836b5934f869fba (table test-table [id=ff3a161f2faf427fb8f51126955f0d1f]) successfully deleted
Sep 25 02:43:37 dist-test-slave-kznw krb5kdc[21556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.13.157.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 02:43:37 dist-test-slave-kznw krb5kdc[21556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.13.157.190@KRBTEST.COM, Server not found in Kerberos database
W20260925 02:43:37.140188 21651 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:43:37.133735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48216 (local address 127.13.157.190:43945)
0925 02:43:37.133873 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:37.133878 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:37.134085 (+ 207us) server_negotiation.cc:407] Connection header received
0925 02:43:37.134191 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:37.134194 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:37.134247 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:37.134339 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:37.134990 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:37.135809 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:37.136594 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:37.136968 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:37.137096 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:37.140061 (+ 2965us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48216: BlockingRecv error: recv got EOF from 127.0.0.1:48216 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Sep 25 02:43:37 dist-test-slave-kznw krb5kdc[21556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304217, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM
I20260925 02:43:37.149912 21651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:37.141521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48224 (local address 127.13.157.190:43945)
0925 02:43:37.141625 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:37.141628 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:37.142028 (+ 400us) server_negotiation.cc:407] Connection header received
0925 02:43:37.142114 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:37.142117 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:37.142164 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:37.142240 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:37.142744 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:37.143554 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:37.144270 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:37.144610 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:37.144702 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:37.146888 (+ 2186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:37.146892 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:37.146895 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:43:37.147043 (+ 148us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:37.148468 (+ 1425us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:37.148827 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:37.148830 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:37.148832 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:37.148916 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:37.149142 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:37.149145 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:37.149147 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:37.149354 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:37.149494 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:37.149641 (+ 147us) server_negotiation.cc:299] Negotiation successful
0925 02:43:37.149752 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260925 02:43:37.152457 21599 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48224:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 02:43:37.152933 21599 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 02:43:37.157773 21740 tablet_service.cc:1467] Processing CreateTablet for tablet b790ae2cca1d400faa966863944b1c9c (DEFAULT_TABLE table=test-table [id=cb5932abb77f4204b83731964fb11212]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:43:37.157791 22023 tablet_service.cc:1467] Processing CreateTablet for tablet b790ae2cca1d400faa966863944b1c9c (DEFAULT_TABLE table=test-table [id=cb5932abb77f4204b83731964fb11212]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:43:37.158077 21740 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b790ae2cca1d400faa966863944b1c9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:43:37.158078 22023 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b790ae2cca1d400faa966863944b1c9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:43:37.160285 22202 tablet_bootstrap.cc:492] T b790ae2cca1d400faa966863944b1c9c P 935a4168b81f4455b6bcf9ba18b1bf80: Bootstrap starting.
I20260925 02:43:37.160928 21882 tablet_service.cc:1467] Processing CreateTablet for tablet b790ae2cca1d400faa966863944b1c9c (DEFAULT_TABLE table=test-table [id=cb5932abb77f4204b83731964fb11212]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:43:37.161206 21882 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b790ae2cca1d400faa966863944b1c9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:43:37.161244 22202 tablet_bootstrap.cc:654] T b790ae2cca1d400faa966863944b1c9c P 935a4168b81f4455b6bcf9ba18b1bf80: Neither blocks nor log segments found. Creating new log.
I20260925 02:43:37.162189 22202 tablet_bootstrap.cc:492] T b790ae2cca1d400faa966863944b1c9c P 935a4168b81f4455b6bcf9ba18b1bf80: No bootstrap required, opened a new log
I20260925 02:43:37.162267 22202 ts_tablet_manager.cc:1397] T b790ae2cca1d400faa966863944b1c9c P 935a4168b81f4455b6bcf9ba18b1bf80: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 02:43:37.162621 22202 raft_consensus.cc:348] T b790ae2cca1d400faa966863944b1c9c P 935a4168b81f4455b6bcf9ba18b1bf80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77e5b44255714c0db6e1fa4e7130c90f" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 40507 } } peers { permanent_uuid: "eb7b2024cae043fd8827ea003fbbcdbf" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33705 } } peers { permanent_uuid: "935a4168b81f4455b6bcf9ba18b1bf80" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 41903 } }
I20260925 02:43:37.162722 22202 raft_consensus.cc:374] T b790ae2cca1d400faa966863944b1c9c P 935a4168b81f4455b6bcf9ba18b1bf80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:43:37.162758 22202 raft_consensus.cc:729] T b790ae2cca1d400faa966863944b1c9c P 935a4168b81f4455b6bcf9ba18b1bf80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 935a4168b81f4455b6bcf9ba18b1bf80, State: Initialized, Role: FOLLOWER
I20260925 02:43:37.162936 22202 consensus_queue.cc:260] T b790ae2cca1d400faa966863944b1c9c P 935a4168b81f4455b6bcf9ba18b1bf80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77e5b44255714c0db6e1fa4e7130c90f" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 40507 } } peers { permanent_uuid: "eb7b2024cae043fd8827ea003fbbcdbf" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33705 } } peers { permanent_uuid: "935a4168b81f4455b6bcf9ba18b1bf80" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 41903 } }
I20260925 02:43:37.164016 22203 tablet_bootstrap.cc:492] T b790ae2cca1d400faa966863944b1c9c P 77e5b44255714c0db6e1fa4e7130c90f: Bootstrap starting.
I20260925 02:43:37.164805 22202 ts_tablet_manager.cc:1428] T b790ae2cca1d400faa966863944b1c9c P 935a4168b81f4455b6bcf9ba18b1bf80: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260925 02:43:37.164999 22203 tablet_bootstrap.cc:654] T b790ae2cca1d400faa966863944b1c9c P 77e5b44255714c0db6e1fa4e7130c90f: Neither blocks nor log segments found. Creating new log.
I20260925 02:43:37.166709 22203 tablet_bootstrap.cc:492] T b790ae2cca1d400faa966863944b1c9c P 77e5b44255714c0db6e1fa4e7130c90f: No bootstrap required, opened a new log
I20260925 02:43:37.166779 22203 ts_tablet_manager.cc:1397] T b790ae2cca1d400faa966863944b1c9c P 77e5b44255714c0db6e1fa4e7130c90f: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260925 02:43:37.167119 22205 tablet_bootstrap.cc:492] T b790ae2cca1d400faa966863944b1c9c P eb7b2024cae043fd8827ea003fbbcdbf: Bootstrap starting.
I20260925 02:43:37.167183 22203 raft_consensus.cc:348] T b790ae2cca1d400faa966863944b1c9c P 77e5b44255714c0db6e1fa4e7130c90f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77e5b44255714c0db6e1fa4e7130c90f" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 40507 } } peers { permanent_uuid: "eb7b2024cae043fd8827ea003fbbcdbf" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33705 } } peers { permanent_uuid: "935a4168b81f4455b6bcf9ba18b1bf80" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 41903 } }
I20260925 02:43:37.167290 22203 raft_consensus.cc:374] T b790ae2cca1d400faa966863944b1c9c P 77e5b44255714c0db6e1fa4e7130c90f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:43:37.167326 22203 raft_consensus.cc:729] T b790ae2cca1d400faa966863944b1c9c P 77e5b44255714c0db6e1fa4e7130c90f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77e5b44255714c0db6e1fa4e7130c90f, State: Initialized, Role: FOLLOWER
I20260925 02:43:37.167451 22203 consensus_queue.cc:260] T b790ae2cca1d400faa966863944b1c9c P 77e5b44255714c0db6e1fa4e7130c90f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77e5b44255714c0db6e1fa4e7130c90f" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 40507 } } peers { permanent_uuid: "eb7b2024cae043fd8827ea003fbbcdbf" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33705 } } peers { permanent_uuid: "935a4168b81f4455b6bcf9ba18b1bf80" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 41903 } }
I20260925 02:43:37.167778 22203 ts_tablet_manager.cc:1428] T b790ae2cca1d400faa966863944b1c9c P 77e5b44255714c0db6e1fa4e7130c90f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260925 02:43:37.167999 22205 tablet_bootstrap.cc:654] T b790ae2cca1d400faa966863944b1c9c P eb7b2024cae043fd8827ea003fbbcdbf: Neither blocks nor log segments found. Creating new log.
I20260925 02:43:37.169893 22205 tablet_bootstrap.cc:492] T b790ae2cca1d400faa966863944b1c9c P eb7b2024cae043fd8827ea003fbbcdbf: No bootstrap required, opened a new log
I20260925 02:43:37.169976 22205 ts_tablet_manager.cc:1397] T b790ae2cca1d400faa966863944b1c9c P eb7b2024cae043fd8827ea003fbbcdbf: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260925 02:43:37.170373 22205 raft_consensus.cc:348] T b790ae2cca1d400faa966863944b1c9c P eb7b2024cae043fd8827ea003fbbcdbf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77e5b44255714c0db6e1fa4e7130c90f" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 40507 } } peers { permanent_uuid: "eb7b2024cae043fd8827ea003fbbcdbf" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33705 } } peers { permanent_uuid: "935a4168b81f4455b6bcf9ba18b1bf80" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 41903 } }
I20260925 02:43:37.170490 22205 raft_consensus.cc:374] T b790ae2cca1d400faa966863944b1c9c P eb7b2024cae043fd8827ea003fbbcdbf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:43:37.170528 22205 raft_consensus.cc:729] T b790ae2cca1d400faa966863944b1c9c P eb7b2024cae043fd8827ea003fbbcdbf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb7b2024cae043fd8827ea003fbbcdbf, State: Initialized, Role: FOLLOWER
I20260925 02:43:37.170645 22205 consensus_queue.cc:260] T b790ae2cca1d400faa966863944b1c9c P eb7b2024cae043fd8827ea003fbbcdbf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77e5b44255714c0db6e1fa4e7130c90f" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 40507 } } peers { permanent_uuid: "eb7b2024cae043fd8827ea003fbbcdbf" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33705 } } peers { permanent_uuid: "935a4168b81f4455b6bcf9ba18b1bf80" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 41903 } }
I20260925 02:43:37.170956 22205 ts_tablet_manager.cc:1428] T b790ae2cca1d400faa966863944b1c9c P eb7b2024cae043fd8827ea003fbbcdbf: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260925 02:43:37.294461 22126 raft_consensus.cc:482] T b790ae2cca1d400faa966863944b1c9c P eb7b2024cae043fd8827ea003fbbcdbf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 02:43:37.294617 22126 raft_consensus.cc:504] T b790ae2cca1d400faa966863944b1c9c P eb7b2024cae043fd8827ea003fbbcdbf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77e5b44255714c0db6e1fa4e7130c90f" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 40507 } } peers { permanent_uuid: "eb7b2024cae043fd8827ea003fbbcdbf" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33705 } } peers { permanent_uuid: "935a4168b81f4455b6bcf9ba18b1bf80" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 41903 } }
I20260925 02:43:37.295058 22126 leader_election.cc:288] T b790ae2cca1d400faa966863944b1c9c P eb7b2024cae043fd8827ea003fbbcdbf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 77e5b44255714c0db6e1fa4e7130c90f (127.13.157.129:40507), 935a4168b81f4455b6bcf9ba18b1bf80 (127.13.157.131:41903)
I20260925 02:43:37.295621 21761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b790ae2cca1d400faa966863944b1c9c" candidate_uuid: "eb7b2024cae043fd8827ea003fbbcdbf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "77e5b44255714c0db6e1fa4e7130c90f" is_pre_election: true
I20260925 02:43:37.295621 22043 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b790ae2cca1d400faa966863944b1c9c" candidate_uuid: "eb7b2024cae043fd8827ea003fbbcdbf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "935a4168b81f4455b6bcf9ba18b1bf80" is_pre_election: true
I20260925 02:43:37.295765 22043 raft_consensus.cc:2454] T b790ae2cca1d400faa966863944b1c9c P 935a4168b81f4455b6bcf9ba18b1bf80 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb7b2024cae043fd8827ea003fbbcdbf in term 0.
I20260925 02:43:37.295774 21761 raft_consensus.cc:2454] T b790ae2cca1d400faa966863944b1c9c P 77e5b44255714c0db6e1fa4e7130c90f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb7b2024cae043fd8827ea003fbbcdbf in term 0.
I20260925 02:43:37.296077 21834 leader_election.cc:302] T b790ae2cca1d400faa966863944b1c9c P eb7b2024cae043fd8827ea003fbbcdbf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 935a4168b81f4455b6bcf9ba18b1bf80, eb7b2024cae043fd8827ea003fbbcdbf; no voters:
I20260925 02:43:37.296276 22126 raft_consensus.cc:2781] T b790ae2cca1d400faa966863944b1c9c P eb7b2024cae043fd8827ea003fbbcdbf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 02:43:37.296336 22126 raft_consensus.cc:482] T b790ae2cca1d400faa966863944b1c9c P eb7b2024cae043fd8827ea003fbbcdbf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 02:43:37.296380 22126 raft_consensus.cc:3037] T b790ae2cca1d400faa966863944b1c9c P eb7b2024cae043fd8827ea003fbbcdbf [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:43:37.297408 22126 raft_consensus.cc:504] T b790ae2cca1d400faa966863944b1c9c P eb7b2024cae043fd8827ea003fbbcdbf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77e5b44255714c0db6e1fa4e7130c90f" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 40507 } } peers { permanent_uuid: "eb7b2024cae043fd8827ea003fbbcdbf" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33705 } } peers { permanent_uuid: "935a4168b81f4455b6bcf9ba18b1bf80" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 41903 } }
I20260925 02:43:37.297724 22126 leader_election.cc:288] T b790ae2cca1d400faa966863944b1c9c P eb7b2024cae043fd8827ea003fbbcdbf [CANDIDATE]: Term 1 election: Requested vote from peers 77e5b44255714c0db6e1fa4e7130c90f (127.13.157.129:40507), 935a4168b81f4455b6bcf9ba18b1bf80 (127.13.157.131:41903)
I20260925 02:43:37.298302 21761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b790ae2cca1d400faa966863944b1c9c" candidate_uuid: "eb7b2024cae043fd8827ea003fbbcdbf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "77e5b44255714c0db6e1fa4e7130c90f"
I20260925 02:43:37.298311 22043 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b790ae2cca1d400faa966863944b1c9c" candidate_uuid: "eb7b2024cae043fd8827ea003fbbcdbf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "935a4168b81f4455b6bcf9ba18b1bf80"
I20260925 02:43:37.298452 22043 raft_consensus.cc:3037] T b790ae2cca1d400faa966863944b1c9c P 935a4168b81f4455b6bcf9ba18b1bf80 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:43:37.298470 21761 raft_consensus.cc:3037] T b790ae2cca1d400faa966863944b1c9c P 77e5b44255714c0db6e1fa4e7130c90f [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:43:37.299743 22043 raft_consensus.cc:2454] T b790ae2cca1d400faa966863944b1c9c P 935a4168b81f4455b6bcf9ba18b1bf80 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb7b2024cae043fd8827ea003fbbcdbf in term 1.
I20260925 02:43:37.299743 21761 raft_consensus.cc:2454] T b790ae2cca1d400faa966863944b1c9c P 77e5b44255714c0db6e1fa4e7130c90f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb7b2024cae043fd8827ea003fbbcdbf in term 1.
I20260925 02:43:37.300066 21834 leader_election.cc:302] T b790ae2cca1d400faa966863944b1c9c P eb7b2024cae043fd8827ea003fbbcdbf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 77e5b44255714c0db6e1fa4e7130c90f, eb7b2024cae043fd8827ea003fbbcdbf; no voters:
I20260925 02:43:37.300236 22126 raft_consensus.cc:2781] T b790ae2cca1d400faa966863944b1c9c P eb7b2024cae043fd8827ea003fbbcdbf [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:43:37.300429 22126 raft_consensus.cc:686] T b790ae2cca1d400faa966863944b1c9c P eb7b2024cae043fd8827ea003fbbcdbf [term 1 LEADER]: Becoming Leader. State: Replica: eb7b2024cae043fd8827ea003fbbcdbf, State: Running, Role: LEADER
I20260925 02:43:37.300697 22126 consensus_queue.cc:237] T b790ae2cca1d400faa966863944b1c9c P eb7b2024cae043fd8827ea003fbbcdbf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77e5b44255714c0db6e1fa4e7130c90f" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 40507 } } peers { permanent_uuid: "eb7b2024cae043fd8827ea003fbbcdbf" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33705 } } peers { permanent_uuid: "935a4168b81f4455b6bcf9ba18b1bf80" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 41903 } }
I20260925 02:43:37.303062 21601 catalog_manager.cc:5495] T b790ae2cca1d400faa966863944b1c9c P eb7b2024cae043fd8827ea003fbbcdbf reported cstate change: term changed from 0 to 1, leader changed from <none> to eb7b2024cae043fd8827ea003fbbcdbf (127.13.157.130). New cstate: current_term: 1 leader_uuid: "eb7b2024cae043fd8827ea003fbbcdbf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77e5b44255714c0db6e1fa4e7130c90f" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 40507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb7b2024cae043fd8827ea003fbbcdbf" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33705 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "935a4168b81f4455b6bcf9ba18b1bf80" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 41903 } health_report { overall_health: UNKNOWN } } }
I20260925 02:43:37.322443 22211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:37.318174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42644 (local address 127.13.157.130:33705)
0925 02:43:37.318467 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:37.318475 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:37.318489 (+ 14us) server_negotiation.cc:407] Connection header received
0925 02:43:37.318572 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:37.318575 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:37.318636 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:37.318770 (+ 134us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 02:43:37.318989 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:37.319847 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:37.320710 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:37.321091 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:37.321184 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:37.321447 (+ 263us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 02:43:37.322068 (+ 621us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 02:43:37.322181 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:37.322212 (+ 31us) server_negotiation.cc:299] Negotiation successful
0925 02:43:37.322266 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":215,"thread_start_us":109,"threads_started":1}
I20260925 02:43:37.325016 21761 raft_consensus.cc:1261] T b790ae2cca1d400faa966863944b1c9c P 77e5b44255714c0db6e1fa4e7130c90f [term 1 FOLLOWER]: Refusing update from remote peer eb7b2024cae043fd8827ea003fbbcdbf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:43:37.325373 22043 raft_consensus.cc:1261] T b790ae2cca1d400faa966863944b1c9c P 935a4168b81f4455b6bcf9ba18b1bf80 [term 1 FOLLOWER]: Refusing update from remote peer eb7b2024cae043fd8827ea003fbbcdbf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:43:37.325425 22208 consensus_queue.cc:1035] T b790ae2cca1d400faa966863944b1c9c P eb7b2024cae043fd8827ea003fbbcdbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "77e5b44255714c0db6e1fa4e7130c90f" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 40507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:43:37.326010 22208 consensus_queue.cc:1035] T b790ae2cca1d400faa966863944b1c9c P eb7b2024cae043fd8827ea003fbbcdbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "935a4168b81f4455b6bcf9ba18b1bf80" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 41903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:43:37.338837 21599 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48224:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 02:43:37.339005 21599 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48224:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 02:43:37.341032 21599 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9120630de98a487eb499476860ce3215: Sending DeleteTablet for 3 replicas of tablet b790ae2cca1d400faa966863944b1c9c
I20260925 02:43:37.341650 21740 tablet_service.cc:1514] Processing DeleteTablet for tablet b790ae2cca1d400faa966863944b1c9c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:43:37 UTC) from {username='oryx', principal='oryx/127.13.157.190@KRBTEST.COM'} at 127.0.0.1:35980
I20260925 02:43:37.341650 22023 tablet_service.cc:1514] Processing DeleteTablet for tablet b790ae2cca1d400faa966863944b1c9c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:43:37 UTC) from {username='oryx', principal='oryx/127.13.157.190@KRBTEST.COM'} at 127.0.0.1:43284
I20260925 02:43:37.341761 21882 tablet_service.cc:1514] Processing DeleteTablet for tablet b790ae2cca1d400faa966863944b1c9c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:43:37 UTC) from {username='oryx', principal='oryx/127.13.157.190@KRBTEST.COM'} at 127.0.0.1:42642
I20260925 02:43:37.341838 22189 tablet_replica.cc:330] T b790ae2cca1d400faa966863944b1c9c P 77e5b44255714c0db6e1fa4e7130c90f: stopping tablet replica
I20260925 02:43:37.341837 22188 tablet_replica.cc:330] T b790ae2cca1d400faa966863944b1c9c P 935a4168b81f4455b6bcf9ba18b1bf80: stopping tablet replica
I20260925 02:43:37.341946 22187 tablet_replica.cc:330] T b790ae2cca1d400faa966863944b1c9c P eb7b2024cae043fd8827ea003fbbcdbf: stopping tablet replica
I20260925 02:43:37.341944 22189 raft_consensus.cc:2229] T b790ae2cca1d400faa966863944b1c9c P 77e5b44255714c0db6e1fa4e7130c90f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 02:43:37.341958 22188 raft_consensus.cc:2229] T b790ae2cca1d400faa966863944b1c9c P 935a4168b81f4455b6bcf9ba18b1bf80 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 02:43:37.342049 22187 raft_consensus.cc:2229] T b790ae2cca1d400faa966863944b1c9c P eb7b2024cae043fd8827ea003fbbcdbf [term 1 LEADER]: Raft consensus shutting down.
I20260925 02:43:37.342084 22188 raft_consensus.cc:2258] T b790ae2cca1d400faa966863944b1c9c P 935a4168b81f4455b6bcf9ba18b1bf80 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:43:37.342137 22189 raft_consensus.cc:2258] T b790ae2cca1d400faa966863944b1c9c P 77e5b44255714c0db6e1fa4e7130c90f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:43:37.342298 22187 raft_consensus.cc:2258] T b790ae2cca1d400faa966863944b1c9c P eb7b2024cae043fd8827ea003fbbcdbf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:43:37.342794 22189 ts_tablet_manager.cc:1905] T b790ae2cca1d400faa966863944b1c9c P 77e5b44255714c0db6e1fa4e7130c90f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:43:37.343012 22187 ts_tablet_manager.cc:1905] T b790ae2cca1d400faa966863944b1c9c P eb7b2024cae043fd8827ea003fbbcdbf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:43:37.343371 21599 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48224:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 02:43:37.343787 21599 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 02:43:37.345244 22187 ts_tablet_manager.cc:1918] T b790ae2cca1d400faa966863944b1c9c P eb7b2024cae043fd8827ea003fbbcdbf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 02:43:37.345244 22189 ts_tablet_manager.cc:1918] T b790ae2cca1d400faa966863944b1c9c P 77e5b44255714c0db6e1fa4e7130c90f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 02:43:37.345321 22189 log.cc:1196] T b790ae2cca1d400faa966863944b1c9c P 77e5b44255714c0db6e1fa4e7130c90f: Deleting WAL directory at /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790304178325258-13942-0/minicluster-data/ts-0/wal/wals/b790ae2cca1d400faa966863944b1c9c
I20260925 02:43:37.345321 22187 log.cc:1196] T b790ae2cca1d400faa966863944b1c9c P eb7b2024cae043fd8827ea003fbbcdbf: Deleting WAL directory at /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790304178325258-13942-0/minicluster-data/ts-1/wal/wals/b790ae2cca1d400faa966863944b1c9c
I20260925 02:43:37.345564 22187 ts_tablet_manager.cc:1939] T b790ae2cca1d400faa966863944b1c9c P eb7b2024cae043fd8827ea003fbbcdbf: Deleting consensus metadata
I20260925 02:43:37.345562 22189 ts_tablet_manager.cc:1939] T b790ae2cca1d400faa966863944b1c9c P 77e5b44255714c0db6e1fa4e7130c90f: Deleting consensus metadata
I20260925 02:43:37.346321 22188 ts_tablet_manager.cc:1905] T b790ae2cca1d400faa966863944b1c9c P 935a4168b81f4455b6bcf9ba18b1bf80: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:43:37.346987 21588 catalog_manager.cc:4841] TS eb7b2024cae043fd8827ea003fbbcdbf (127.13.157.130:33705): tablet b790ae2cca1d400faa966863944b1c9c (table test-table [id=cb5932abb77f4204b83731964fb11212]) successfully deleted
I20260925 02:43:37.347213 21586 catalog_manager.cc:4841] TS 77e5b44255714c0db6e1fa4e7130c90f (127.13.157.129:40507): tablet b790ae2cca1d400faa966863944b1c9c (table test-table [id=cb5932abb77f4204b83731964fb11212]) successfully deleted
I20260925 02:43:37.348878 22188 ts_tablet_manager.cc:1918] T b790ae2cca1d400faa966863944b1c9c P 935a4168b81f4455b6bcf9ba18b1bf80: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 02:43:37.348955 22188 log.cc:1196] T b790ae2cca1d400faa966863944b1c9c P 935a4168b81f4455b6bcf9ba18b1bf80: Deleting WAL directory at /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790304178325258-13942-0/minicluster-data/ts-2/wal/wals/b790ae2cca1d400faa966863944b1c9c
I20260925 02:43:37.348970 21882 tablet_service.cc:1467] Processing CreateTablet for tablet 15e1c326a5f44d64af38ac41f23af03e (DEFAULT_TABLE table=test-table [id=b87f5ca245de4f0390fd8f0d25c84937]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:43:37.349010 21740 tablet_service.cc:1467] Processing CreateTablet for tablet 15e1c326a5f44d64af38ac41f23af03e (DEFAULT_TABLE table=test-table [id=b87f5ca245de4f0390fd8f0d25c84937]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:43:37.349193 22188 ts_tablet_manager.cc:1939] T b790ae2cca1d400faa966863944b1c9c P 935a4168b81f4455b6bcf9ba18b1bf80: Deleting consensus metadata
I20260925 02:43:37.349206 21882 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 15e1c326a5f44d64af38ac41f23af03e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:43:37.349246 21740 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 15e1c326a5f44d64af38ac41f23af03e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:43:37.350111 22023 tablet_service.cc:1467] Processing CreateTablet for tablet 15e1c326a5f44d64af38ac41f23af03e (DEFAULT_TABLE table=test-table [id=b87f5ca245de4f0390fd8f0d25c84937]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:43:37.350332 22023 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 15e1c326a5f44d64af38ac41f23af03e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:43:37.351405 22205 tablet_bootstrap.cc:492] T 15e1c326a5f44d64af38ac41f23af03e P eb7b2024cae043fd8827ea003fbbcdbf: Bootstrap starting.
I20260925 02:43:37.352325 22205 tablet_bootstrap.cc:654] T 15e1c326a5f44d64af38ac41f23af03e P eb7b2024cae043fd8827ea003fbbcdbf: Neither blocks nor log segments found. Creating new log.
I20260925 02:43:37.352398 22202 tablet_bootstrap.cc:492] T 15e1c326a5f44d64af38ac41f23af03e P 935a4168b81f4455b6bcf9ba18b1bf80: Bootstrap starting.
I20260925 02:43:37.353147 22202 tablet_bootstrap.cc:654] T 15e1c326a5f44d64af38ac41f23af03e P 935a4168b81f4455b6bcf9ba18b1bf80: Neither blocks nor log segments found. Creating new log.
I20260925 02:43:37.353281 22203 tablet_bootstrap.cc:492] T 15e1c326a5f44d64af38ac41f23af03e P 77e5b44255714c0db6e1fa4e7130c90f: Bootstrap starting.
I20260925 02:43:37.353281 21586 catalog_manager.cc:4841] TS 935a4168b81f4455b6bcf9ba18b1bf80 (127.13.157.131:41903): tablet b790ae2cca1d400faa966863944b1c9c (table test-table [id=cb5932abb77f4204b83731964fb11212]) successfully deleted
I20260925 02:43:37.353842 22205 tablet_bootstrap.cc:492] T 15e1c326a5f44d64af38ac41f23af03e P eb7b2024cae043fd8827ea003fbbcdbf: No bootstrap required, opened a new log
I20260925 02:43:37.353929 22205 ts_tablet_manager.cc:1397] T 15e1c326a5f44d64af38ac41f23af03e P eb7b2024cae043fd8827ea003fbbcdbf: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260925 02:43:37.354075 22203 tablet_bootstrap.cc:654] T 15e1c326a5f44d64af38ac41f23af03e P 77e5b44255714c0db6e1fa4e7130c90f: Neither blocks nor log segments found. Creating new log.
I20260925 02:43:37.354391 22205 raft_consensus.cc:348] T 15e1c326a5f44d64af38ac41f23af03e P eb7b2024cae043fd8827ea003fbbcdbf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77e5b44255714c0db6e1fa4e7130c90f" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 40507 } } peers { permanent_uuid: "eb7b2024cae043fd8827ea003fbbcdbf" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33705 } } peers { permanent_uuid: "935a4168b81f4455b6bcf9ba18b1bf80" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 41903 } }
I20260925 02:43:37.354526 22205 raft_consensus.cc:374] T 15e1c326a5f44d64af38ac41f23af03e P eb7b2024cae043fd8827ea003fbbcdbf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:43:37.354569 22205 raft_consensus.cc:729] T 15e1c326a5f44d64af38ac41f23af03e P eb7b2024cae043fd8827ea003fbbcdbf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb7b2024cae043fd8827ea003fbbcdbf, State: Initialized, Role: FOLLOWER
I20260925 02:43:37.354689 22205 consensus_queue.cc:260] T 15e1c326a5f44d64af38ac41f23af03e P eb7b2024cae043fd8827ea003fbbcdbf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77e5b44255714c0db6e1fa4e7130c90f" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 40507 } } peers { permanent_uuid: "eb7b2024cae043fd8827ea003fbbcdbf" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33705 } } peers { permanent_uuid: "935a4168b81f4455b6bcf9ba18b1bf80" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 41903 } }
I20260925 02:43:37.355312 22203 tablet_bootstrap.cc:492] T 15e1c326a5f44d64af38ac41f23af03e P 77e5b44255714c0db6e1fa4e7130c90f: No bootstrap required, opened a new log
I20260925 02:43:37.355389 22202 tablet_bootstrap.cc:492] T 15e1c326a5f44d64af38ac41f23af03e P 935a4168b81f4455b6bcf9ba18b1bf80: No bootstrap required, opened a new log
I20260925 02:43:37.355474 22202 ts_tablet_manager.cc:1397] T 15e1c326a5f44d64af38ac41f23af03e P 935a4168b81f4455b6bcf9ba18b1bf80: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260925 02:43:37.355474 22203 ts_tablet_manager.cc:1397] T 15e1c326a5f44d64af38ac41f23af03e P 77e5b44255714c0db6e1fa4e7130c90f: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260925 02:43:37.355556 22205 ts_tablet_manager.cc:1428] T 15e1c326a5f44d64af38ac41f23af03e P eb7b2024cae043fd8827ea003fbbcdbf: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260925 02:43:37.355850 22202 raft_consensus.cc:348] T 15e1c326a5f44d64af38ac41f23af03e P 935a4168b81f4455b6bcf9ba18b1bf80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77e5b44255714c0db6e1fa4e7130c90f" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 40507 } } peers { permanent_uuid: "eb7b2024cae043fd8827ea003fbbcdbf" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33705 } } peers { permanent_uuid: "935a4168b81f4455b6bcf9ba18b1bf80" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 41903 } }
I20260925 02:43:37.355962 22202 raft_consensus.cc:374] T 15e1c326a5f44d64af38ac41f23af03e P 935a4168b81f4455b6bcf9ba18b1bf80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:43:37.356002 22202 raft_consensus.cc:729] T 15e1c326a5f44d64af38ac41f23af03e P 935a4168b81f4455b6bcf9ba18b1bf80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 935a4168b81f4455b6bcf9ba18b1bf80, State: Initialized, Role: FOLLOWER
I20260925 02:43:37.355922 22203 raft_consensus.cc:348] T 15e1c326a5f44d64af38ac41f23af03e P 77e5b44255714c0db6e1fa4e7130c90f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77e5b44255714c0db6e1fa4e7130c90f" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 40507 } } peers { permanent_uuid: "eb7b2024cae043fd8827ea003fbbcdbf" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33705 } } peers { permanent_uuid: "935a4168b81f4455b6bcf9ba18b1bf80" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 41903 } }
I20260925 02:43:37.356141 22202 consensus_queue.cc:260] T 15e1c326a5f44d64af38ac41f23af03e P 935a4168b81f4455b6bcf9ba18b1bf80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77e5b44255714c0db6e1fa4e7130c90f" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 40507 } } peers { permanent_uuid: "eb7b2024cae043fd8827ea003fbbcdbf" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33705 } } peers { permanent_uuid: "935a4168b81f4455b6bcf9ba18b1bf80" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 41903 } }
I20260925 02:43:37.356396 22202 ts_tablet_manager.cc:1428] T 15e1c326a5f44d64af38ac41f23af03e P 935a4168b81f4455b6bcf9ba18b1bf80: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260925 02:43:37.356225 22203 raft_consensus.cc:374] T 15e1c326a5f44d64af38ac41f23af03e P 77e5b44255714c0db6e1fa4e7130c90f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:43:37.357465 22203 raft_consensus.cc:729] T 15e1c326a5f44d64af38ac41f23af03e P 77e5b44255714c0db6e1fa4e7130c90f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77e5b44255714c0db6e1fa4e7130c90f, State: Initialized, Role: FOLLOWER
I20260925 02:43:37.357630 22203 consensus_queue.cc:260] T 15e1c326a5f44d64af38ac41f23af03e P 77e5b44255714c0db6e1fa4e7130c90f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77e5b44255714c0db6e1fa4e7130c90f" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 40507 } } peers { permanent_uuid: "eb7b2024cae043fd8827ea003fbbcdbf" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33705 } } peers { permanent_uuid: "935a4168b81f4455b6bcf9ba18b1bf80" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 41903 } }
I20260925 02:43:37.357980 22203 ts_tablet_manager.cc:1428] T 15e1c326a5f44d64af38ac41f23af03e P 77e5b44255714c0db6e1fa4e7130c90f: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260925 02:43:37.416707 22181 raft_consensus.cc:482] T 15e1c326a5f44d64af38ac41f23af03e P 935a4168b81f4455b6bcf9ba18b1bf80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 02:43:37.416862 22181 raft_consensus.cc:504] T 15e1c326a5f44d64af38ac41f23af03e P 935a4168b81f4455b6bcf9ba18b1bf80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77e5b44255714c0db6e1fa4e7130c90f" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 40507 } } peers { permanent_uuid: "eb7b2024cae043fd8827ea003fbbcdbf" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33705 } } peers { permanent_uuid: "935a4168b81f4455b6bcf9ba18b1bf80" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 41903 } }
I20260925 02:43:37.417349 22181 leader_election.cc:288] T 15e1c326a5f44d64af38ac41f23af03e P 935a4168b81f4455b6bcf9ba18b1bf80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 77e5b44255714c0db6e1fa4e7130c90f (127.13.157.129:40507), eb7b2024cae043fd8827ea003fbbcdbf (127.13.157.130:33705)
I20260925 02:43:37.417829 21761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15e1c326a5f44d64af38ac41f23af03e" candidate_uuid: "935a4168b81f4455b6bcf9ba18b1bf80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "77e5b44255714c0db6e1fa4e7130c90f" is_pre_election: true
I20260925 02:43:37.417932 21902 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15e1c326a5f44d64af38ac41f23af03e" candidate_uuid: "935a4168b81f4455b6bcf9ba18b1bf80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb7b2024cae043fd8827ea003fbbcdbf" is_pre_election: true
I20260925 02:43:37.418004 21761 raft_consensus.cc:2454] T 15e1c326a5f44d64af38ac41f23af03e P 77e5b44255714c0db6e1fa4e7130c90f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 935a4168b81f4455b6bcf9ba18b1bf80 in term 0.
I20260925 02:43:37.418092 21902 raft_consensus.cc:2454] T 15e1c326a5f44d64af38ac41f23af03e P eb7b2024cae043fd8827ea003fbbcdbf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 935a4168b81f4455b6bcf9ba18b1bf80 in term 0.
I20260925 02:43:37.418365 21974 leader_election.cc:302] T 15e1c326a5f44d64af38ac41f23af03e P 935a4168b81f4455b6bcf9ba18b1bf80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 77e5b44255714c0db6e1fa4e7130c90f, 935a4168b81f4455b6bcf9ba18b1bf80; no voters:
I20260925 02:43:37.418560 22181 raft_consensus.cc:2781] T 15e1c326a5f44d64af38ac41f23af03e P 935a4168b81f4455b6bcf9ba18b1bf80 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 02:43:37.418630 22181 raft_consensus.cc:482] T 15e1c326a5f44d64af38ac41f23af03e P 935a4168b81f4455b6bcf9ba18b1bf80 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 02:43:37.418676 22181 raft_consensus.cc:3037] T 15e1c326a5f44d64af38ac41f23af03e P 935a4168b81f4455b6bcf9ba18b1bf80 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:43:37.419719 22181 raft_consensus.cc:504] T 15e1c326a5f44d64af38ac41f23af03e P 935a4168b81f4455b6bcf9ba18b1bf80 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77e5b44255714c0db6e1fa4e7130c90f" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 40507 } } peers { permanent_uuid: "eb7b2024cae043fd8827ea003fbbcdbf" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33705 } } peers { permanent_uuid: "935a4168b81f4455b6bcf9ba18b1bf80" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 41903 } }
I20260925 02:43:37.420027 22181 leader_election.cc:288] T 15e1c326a5f44d64af38ac41f23af03e P 935a4168b81f4455b6bcf9ba18b1bf80 [CANDIDATE]: Term 1 election: Requested vote from peers 77e5b44255714c0db6e1fa4e7130c90f (127.13.157.129:40507), eb7b2024cae043fd8827ea003fbbcdbf (127.13.157.130:33705)
I20260925 02:43:37.420378 21902 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15e1c326a5f44d64af38ac41f23af03e" candidate_uuid: "935a4168b81f4455b6bcf9ba18b1bf80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb7b2024cae043fd8827ea003fbbcdbf"
I20260925 02:43:37.420400 21761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15e1c326a5f44d64af38ac41f23af03e" candidate_uuid: "935a4168b81f4455b6bcf9ba18b1bf80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "77e5b44255714c0db6e1fa4e7130c90f"
I20260925 02:43:37.420493 21902 raft_consensus.cc:3037] T 15e1c326a5f44d64af38ac41f23af03e P eb7b2024cae043fd8827ea003fbbcdbf [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:43:37.420493 21761 raft_consensus.cc:3037] T 15e1c326a5f44d64af38ac41f23af03e P 77e5b44255714c0db6e1fa4e7130c90f [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:43:37.421591 21902 raft_consensus.cc:2454] T 15e1c326a5f44d64af38ac41f23af03e P eb7b2024cae043fd8827ea003fbbcdbf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 935a4168b81f4455b6bcf9ba18b1bf80 in term 1.
I20260925 02:43:37.421591 21761 raft_consensus.cc:2454] T 15e1c326a5f44d64af38ac41f23af03e P 77e5b44255714c0db6e1fa4e7130c90f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 935a4168b81f4455b6bcf9ba18b1bf80 in term 1.
I20260925 02:43:37.421878 21974 leader_election.cc:302] T 15e1c326a5f44d64af38ac41f23af03e P 935a4168b81f4455b6bcf9ba18b1bf80 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 77e5b44255714c0db6e1fa4e7130c90f, 935a4168b81f4455b6bcf9ba18b1bf80; no voters:
I20260925 02:43:37.422045 22181 raft_consensus.cc:2781] T 15e1c326a5f44d64af38ac41f23af03e P 935a4168b81f4455b6bcf9ba18b1bf80 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:43:37.422133 22181 raft_consensus.cc:686] T 15e1c326a5f44d64af38ac41f23af03e P 935a4168b81f4455b6bcf9ba18b1bf80 [term 1 LEADER]: Becoming Leader. State: Replica: 935a4168b81f4455b6bcf9ba18b1bf80, State: Running, Role: LEADER
I20260925 02:43:37.422269 22181 consensus_queue.cc:237] T 15e1c326a5f44d64af38ac41f23af03e P 935a4168b81f4455b6bcf9ba18b1bf80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77e5b44255714c0db6e1fa4e7130c90f" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 40507 } } peers { permanent_uuid: "eb7b2024cae043fd8827ea003fbbcdbf" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33705 } } peers { permanent_uuid: "935a4168b81f4455b6bcf9ba18b1bf80" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 41903 } }
I20260925 02:43:37.423962 21598 catalog_manager.cc:5495] T 15e1c326a5f44d64af38ac41f23af03e P 935a4168b81f4455b6bcf9ba18b1bf80 reported cstate change: term changed from 0 to 1, leader changed from <none> to 935a4168b81f4455b6bcf9ba18b1bf80 (127.13.157.131). New cstate: current_term: 1 leader_uuid: "935a4168b81f4455b6bcf9ba18b1bf80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77e5b44255714c0db6e1fa4e7130c90f" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 40507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb7b2024cae043fd8827ea003fbbcdbf" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33705 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "935a4168b81f4455b6bcf9ba18b1bf80" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 41903 } health_report { overall_health: HEALTHY } } }
I20260925 02:43:37.446763 22117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:37.443186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43304 (local address 127.13.157.131:41903)
0925 02:43:37.443300 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:37.443304 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:37.443320 (+ 16us) server_negotiation.cc:407] Connection header received
0925 02:43:37.443419 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:37.443422 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:37.443473 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:37.443561 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 02:43:37.443744 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:37.444496 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:37.445509 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:37.445853 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:37.445976 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:37.446254 (+ 278us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 02:43:37.446342 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 02:43:37.446421 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:37.446562 (+ 141us) server_negotiation.cc:299] Negotiation successful
0925 02:43:37.446623 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260925 02:43:37.455210 22117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:37.452122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.131:54001 (local address 127.13.157.131:41903)
0925 02:43:37.452218 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:37.452221 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:37.452232 (+ 11us) server_negotiation.cc:407] Connection header received
0925 02:43:37.452377 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:37.452380 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:37.452441 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:37.452499 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 02:43:37.452722 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:37.453189 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:37.454274 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:37.454469 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:37.454546 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:37.454791 (+ 245us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 02:43:37.454847 (+ 56us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 02:43:37.454891 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:37.455034 (+ 143us) server_negotiation.cc:299] Negotiation successful
0925 02:43:37.455080 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":35}
I20260925 02:43:37.455243 22184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:37.452001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.131:41903 (local address 127.13.157.131:54001)
0925 02:43:37.452172 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:37.452186 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:37.452292 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:37.452534 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:37.452537 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:37.452546 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 02:43:37.452639 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:37.452645 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:37.453296 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:37.453299 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:37.454168 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:37.454179 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:37.454578 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:37.454582 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:37.454665 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:37.454695 (+ 30us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 02:43:37.454944 (+ 249us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 02:43:37.454952 (+ 8us) client_negotiation.cc:769] Sending connection context
0925 02:43:37.455022 (+ 70us) client_negotiation.cc:240] Negotiation successful
0925 02:43:37.455102 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92}
I20260925 02:43:37.460803 21761 raft_consensus.cc:1261] T 15e1c326a5f44d64af38ac41f23af03e P 77e5b44255714c0db6e1fa4e7130c90f [term 1 FOLLOWER]: Refusing update from remote peer 935a4168b81f4455b6bcf9ba18b1bf80: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:43:37.461046 21902 raft_consensus.cc:1261] T 15e1c326a5f44d64af38ac41f23af03e P eb7b2024cae043fd8827ea003fbbcdbf [term 1 FOLLOWER]: Refusing update from remote peer 935a4168b81f4455b6bcf9ba18b1bf80: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:43:37.461418 22128 consensus_queue.cc:1035] T 15e1c326a5f44d64af38ac41f23af03e P 935a4168b81f4455b6bcf9ba18b1bf80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb7b2024cae043fd8827ea003fbbcdbf" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:43:37.461640 22181 consensus_queue.cc:1035] T 15e1c326a5f44d64af38ac41f23af03e P 935a4168b81f4455b6bcf9ba18b1bf80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "77e5b44255714c0db6e1fa4e7130c90f" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 40507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:43:37.494078 21599 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48224:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 02:43:37.494231 21599 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48224:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 02:43:37.496122 21599 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9120630de98a487eb499476860ce3215: Sending DeleteTablet for 3 replicas of tablet 15e1c326a5f44d64af38ac41f23af03e
I20260925 02:43:37.496790 21882 tablet_service.cc:1514] Processing DeleteTablet for tablet 15e1c326a5f44d64af38ac41f23af03e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:43:37 UTC) from {username='oryx', principal='oryx/127.13.157.190@KRBTEST.COM'} at 127.0.0.1:42642
I20260925 02:43:37.496822 22022 tablet_service.cc:1514] Processing DeleteTablet for tablet 15e1c326a5f44d64af38ac41f23af03e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:43:37 UTC) from {username='oryx', principal='oryx/127.13.157.190@KRBTEST.COM'} at 127.0.0.1:43284
I20260925 02:43:37.496980 21740 tablet_service.cc:1514] Processing DeleteTablet for tablet 15e1c326a5f44d64af38ac41f23af03e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:43:37 UTC) from {username='oryx', principal='oryx/127.13.157.190@KRBTEST.COM'} at 127.0.0.1:35980
I20260925 02:43:37.497021 22188 tablet_replica.cc:330] T 15e1c326a5f44d64af38ac41f23af03e P 935a4168b81f4455b6bcf9ba18b1bf80: stopping tablet replica
I20260925 02:43:37.497144 22188 raft_consensus.cc:2229] T 15e1c326a5f44d64af38ac41f23af03e P 935a4168b81f4455b6bcf9ba18b1bf80 [term 1 LEADER]: Raft consensus shutting down.
I20260925 02:43:37.497169 22187 tablet_replica.cc:330] T 15e1c326a5f44d64af38ac41f23af03e P eb7b2024cae043fd8827ea003fbbcdbf: stopping tablet replica
I20260925 02:43:37.497273 22187 raft_consensus.cc:2229] T 15e1c326a5f44d64af38ac41f23af03e P eb7b2024cae043fd8827ea003fbbcdbf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 02:43:37.497323 22188 raft_consensus.cc:2258] T 15e1c326a5f44d64af38ac41f23af03e P 935a4168b81f4455b6bcf9ba18b1bf80 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:43:37.497387 22187 raft_consensus.cc:2258] T 15e1c326a5f44d64af38ac41f23af03e P eb7b2024cae043fd8827ea003fbbcdbf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:43:37.497560 22189 tablet_replica.cc:330] T 15e1c326a5f44d64af38ac41f23af03e P 77e5b44255714c0db6e1fa4e7130c90f: stopping tablet replica
I20260925 02:43:37.497679 22189 raft_consensus.cc:2229] T 15e1c326a5f44d64af38ac41f23af03e P 77e5b44255714c0db6e1fa4e7130c90f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 02:43:37.497792 22189 raft_consensus.cc:2258] T 15e1c326a5f44d64af38ac41f23af03e P 77e5b44255714c0db6e1fa4e7130c90f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:43:37.497927 22188 ts_tablet_manager.cc:1905] T 15e1c326a5f44d64af38ac41f23af03e P 935a4168b81f4455b6bcf9ba18b1bf80: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:43:37.498699 22189 ts_tablet_manager.cc:1905] T 15e1c326a5f44d64af38ac41f23af03e P 77e5b44255714c0db6e1fa4e7130c90f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:43:37.498713 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 21674
I20260925 02:43:37.500388 22188 ts_tablet_manager.cc:1918] T 15e1c326a5f44d64af38ac41f23af03e P 935a4168b81f4455b6bcf9ba18b1bf80: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 02:43:37.500447 22188 log.cc:1196] T 15e1c326a5f44d64af38ac41f23af03e P 935a4168b81f4455b6bcf9ba18b1bf80: Deleting WAL directory at /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790304178325258-13942-0/minicluster-data/ts-2/wal/wals/15e1c326a5f44d64af38ac41f23af03e
I20260925 02:43:37.500480 22187 ts_tablet_manager.cc:1905] T 15e1c326a5f44d64af38ac41f23af03e P eb7b2024cae043fd8827ea003fbbcdbf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:43:37.500664 22188 ts_tablet_manager.cc:1939] T 15e1c326a5f44d64af38ac41f23af03e P 935a4168b81f4455b6bcf9ba18b1bf80: Deleting consensus metadata
I20260925 02:43:37.501987 21586 catalog_manager.cc:4841] TS 935a4168b81f4455b6bcf9ba18b1bf80 (127.13.157.131:41903): tablet 15e1c326a5f44d64af38ac41f23af03e (table test-table [id=b87f5ca245de4f0390fd8f0d25c84937]) successfully deleted
I20260925 02:43:37.502794 22187 ts_tablet_manager.cc:1918] T 15e1c326a5f44d64af38ac41f23af03e P eb7b2024cae043fd8827ea003fbbcdbf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 02:43:37.502868 22187 log.cc:1196] T 15e1c326a5f44d64af38ac41f23af03e P eb7b2024cae043fd8827ea003fbbcdbf: Deleting WAL directory at /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790304178325258-13942-0/minicluster-data/ts-1/wal/wals/15e1c326a5f44d64af38ac41f23af03e
I20260925 02:43:37.503123 22187 ts_tablet_manager.cc:1939] T 15e1c326a5f44d64af38ac41f23af03e P eb7b2024cae043fd8827ea003fbbcdbf: Deleting consensus metadata
I20260925 02:43:37.503731 21588 catalog_manager.cc:4841] TS eb7b2024cae043fd8827ea003fbbcdbf (127.13.157.130:33705): tablet 15e1c326a5f44d64af38ac41f23af03e (table test-table [id=b87f5ca245de4f0390fd8f0d25c84937]) successfully deleted
W20260925 02:43:37.507798 21586 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.157.129:40507 (error 108)
I20260925 02:43:37.507911 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 21816
I20260925 02:43:37.512583 22226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:37.512231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.129:40507 (local address 127.0.0.1:35990)
0925 02:43:37.512426 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:37.512497 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.129:40507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":55,"threads_started":1}
W20260925 02:43:37.512705 21586 catalog_manager.cc:4584] TS 77e5b44255714c0db6e1fa4e7130c90f (127.13.157.129:40507): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 15e1c326a5f44d64af38ac41f23af03e: Network error: Client connection negotiation failed: client connection to 127.13.157.129:40507: connect: Connection refused (error 111)
I20260925 02:43:37.514746 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 21957
I20260925 02:43:37.520488 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 21569
2026-09-25T02:43:37Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3182 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790304178325258-13942-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 02:43:37 dist-test-slave-kznw krb5kdc[22229](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 02:43:37 dist-test-slave-kznw krb5kdc[22229](info): set up 2 sockets
Sep 25 02:43:37 dist-test-slave-kznw krb5kdc[22229](info): commencing operation
krb5kdc: starting...
W20260925 02:43:38.478937 13942 mini_kdc.cc:121] Time spent starting KDC: real 0.937s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790304178325258-13942-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790304178325258-13942-0/krb5kdc/test-user.keytab.
Sep 25 02:43:38 dist-test-slave-kznw krb5kdc[22229](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304218, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T02:43:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T02:43:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.157.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.157.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.157.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab.
Entry for principal kudu/127.13.157.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab.
I20260925 02:43:38.595283 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790304178325258-13942-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790304178325258-13942-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790304178325258-13942-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.157.190:42103
--webserver_interface=127.13.157.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab
--principal=kudu/127.13.157.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:37433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.157.190:42103
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:43:38.703083 22242 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:43:38.703332 22242 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:43:38.703387 22242 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:43:38.707284 22242 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:43:38.707334 22242 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:43:38.707360 22242 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:43:38.707383 22242 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:43:38.707412 22242 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:43:38.712477 22242 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:37433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790304178325258-13942-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790304178325258-13942-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.157.190:42103
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.157.190:42103
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.157.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22242
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790304178325258-13942-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:43:38.713459 22242 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:43:38.714447 22242 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:43:38.722664 22250 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:38.722678 22248 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:38.722731 22247 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:43:38.722724 22242 server_base.cc:1034] running on GCE node
I20260925 02:43:38.723560 22242 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:43:38.724548 22242 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:43:38.725721 22242 hybrid_clock.cc:648] HybridClock initialized: now 1790304218725707 us; error 25 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-09-25T02:43:38Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1212 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 02:43:38 dist-test-slave-kznw krb5kdc[22255](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 02:43:38 dist-test-slave-kznw krb5kdc[22255](info): set up 2 sockets
Sep 25 02:43:38 dist-test-slave-kznw krb5kdc[22255](info): commencing operation
krb5kdc: starting...
W20260925 02:43:39.695050 13942 mini_kdc.cc:121] Time spent starting KDC: real 0.940s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/krb5kdc/test-user.keytab.
Sep 25 02:43:39 dist-test-slave-kznw krb5kdc[22255](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304219, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T02:43:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T02:43:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.157.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.157.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.157.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab.
Entry for principal kudu/127.13.157.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab.
I20260925 02:43:39.813824 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.157.190:45019
--webserver_interface=127.13.157.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab
--principal=kudu/127.13.157.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:40879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.157.190:45019
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:43:39.923579 22268 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:43:39.923841 22268 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:43:39.923882 22268 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:43:39.927894 22268 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:43:39.927955 22268 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:43:39.927974 22268 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:43:39.928031 22268 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:43:39.928062 22268 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:43:39.933422 22268 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:40879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.157.190:45019
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab
--principal=kudu/127.13.157.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.157.190:45019
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.157.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22268
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:43:39.934515 22268 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:43:39.935564 22268 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:43:39.943846 22273 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:39.943848 22276 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:39.943838 22274 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:43:39.944096 22268 server_base.cc:1034] running on GCE node
I20260925 02:43:39.944748 22268 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:43:39.945973 22268 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:43:39.947160 22268 hybrid_clock.cc:648] HybridClock initialized: now 1790304219947142 us; error 30 us; skew 500 ppm
Sep 25 02:43:39 dist-test-slave-kznw krb5kdc[22255](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304219, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:43:39.950008 22268 init.cc:377] Logged in from keytab as kudu/127.13.157.190@KRBTEST.COM (short username kudu)
I20260925 02:43:39.951038 22268 webserver.cc:460] Webserver started at http://127.13.157.190:45455/ using document root <none> and password file <none>
I20260925 02:43:39.951596 22268 fs_manager.cc:359] Metadata directory not provided
I20260925 02:43:39.951668 22268 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:43:39.951870 22268 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:43:39.953979 22268 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/master-0/data/instance:
uuid: "9055c7c9854247da9baefe43e573e9e2"
format_stamp: "Formatted at 2026-09-25 02:43:39 on dist-test-slave-kznw"
server_key: "1a82ca68cc8f49584cf9c6043689ff74"
server_key_iv: "93c886619de6080c709b0ffcbad2559a"
server_key_version: "encryptionkey@0"
I20260925 02:43:39.954531 22268 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/master-0/wal/instance:
uuid: "9055c7c9854247da9baefe43e573e9e2"
format_stamp: "Formatted at 2026-09-25 02:43:39 on dist-test-slave-kznw"
server_key: "1a82ca68cc8f49584cf9c6043689ff74"
server_key_iv: "93c886619de6080c709b0ffcbad2559a"
server_key_version: "encryptionkey@0"
I20260925 02:43:39.958302 22268 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260925 02:43:39.960711 22283 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:39.961983 22268 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260925 02:43:39.962157 22268 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/master-0/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/master-0/wal
uuid: "9055c7c9854247da9baefe43e573e9e2"
format_stamp: "Formatted at 2026-09-25 02:43:39 on dist-test-slave-kznw"
server_key: "1a82ca68cc8f49584cf9c6043689ff74"
server_key_iv: "93c886619de6080c709b0ffcbad2559a"
server_key_version: "encryptionkey@0"
I20260925 02:43:39.962261 22268 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:43:39.974525 22268 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:43:39.977454 22268 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:43:39.977638 22268 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:43:39.984558 22268 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.190:45019
I20260925 02:43:39.984591 22335 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.190:45019 every 8 connection(s)
I20260925 02:43:39.985523 22268 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
I20260925 02:43:39.988296 22336 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:43:39.989677 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 22268
I20260925 02:43:39.989814 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/master-0/wal/instance
I20260925 02:43:39.990144 13942 external_mini_cluster.cc:1428] Setting key 30a8e042e6a5637266d3ec2e1ca3d55e
I20260925 02:43:39.994412 22336 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9055c7c9854247da9baefe43e573e9e2: Bootstrap starting.
I20260925 02:43:39.996605 22336 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9055c7c9854247da9baefe43e573e9e2: Neither blocks nor log segments found. Creating new log.
I20260925 02:43:39.997433 22336 log.cc:824] T 00000000000000000000000000000000 P 9055c7c9854247da9baefe43e573e9e2: Log is configured to *not* fsync() on all Append() calls
Sep 25 02:43:39 dist-test-slave-kznw krb5kdc[22255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304219, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.157.190@KRBTEST.COM
I20260925 02:43:39.999410 22336 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9055c7c9854247da9baefe43e573e9e2: No bootstrap required, opened a new log
I20260925 02:43:40.002889 22336 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9055c7c9854247da9baefe43e573e9e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9055c7c9854247da9baefe43e573e9e2" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 45019 } }
I20260925 02:43:40.003187 22336 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9055c7c9854247da9baefe43e573e9e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:43:40.003252 22336 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9055c7c9854247da9baefe43e573e9e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9055c7c9854247da9baefe43e573e9e2, State: Initialized, Role: FOLLOWER
I20260925 02:43:40.003679 22336 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9055c7c9854247da9baefe43e573e9e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9055c7c9854247da9baefe43e573e9e2" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 45019 } }
I20260925 02:43:40.003801 22336 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9055c7c9854247da9baefe43e573e9e2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 02:43:40.003862 22336 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9055c7c9854247da9baefe43e573e9e2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 02:43:40.003949 22339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:39.991673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47828 (local address 127.13.157.190:45019)
0925 02:43:39.992129 (+ 456us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:39.992145 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:39.992183 (+ 38us) server_negotiation.cc:407] Connection header received
0925 02:43:39.992828 (+ 645us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:39.992846 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:39.993170 (+ 324us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:39.993499 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:39.994266 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:39.995311 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:39.996164 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:39.996630 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:39.996787 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:39.999399 (+ 2612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:39.999404 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:39.999417 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:43:39.999444 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:40.001147 (+ 1703us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:40.001581 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:40.001585 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:40.001590 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:40.001699 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:40.001946 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:40.001949 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:40.001951 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:40.002333 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:40.002519 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:40.002798 (+ 279us) server_negotiation.cc:299] Negotiation successful
0925 02:43:40.003101 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":132,"threads_started":1}
I20260925 02:43:40.003950 22336 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9055c7c9854247da9baefe43e573e9e2 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:43:40.004870 22336 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9055c7c9854247da9baefe43e573e9e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9055c7c9854247da9baefe43e573e9e2" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 45019 } }
I20260925 02:43:40.006158 22336 leader_election.cc:302] T 00000000000000000000000000000000 P 9055c7c9854247da9baefe43e573e9e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9055c7c9854247da9baefe43e573e9e2; no voters:
I20260925 02:43:40.006431 22336 leader_election.cc:288] T 00000000000000000000000000000000 P 9055c7c9854247da9baefe43e573e9e2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 02:43:40.006551 22341 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9055c7c9854247da9baefe43e573e9e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:43:40.006804 22341 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9055c7c9854247da9baefe43e573e9e2 [term 1 LEADER]: Becoming Leader. State: Replica: 9055c7c9854247da9baefe43e573e9e2, State: Running, Role: LEADER
I20260925 02:43:40.007264 22341 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9055c7c9854247da9baefe43e573e9e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9055c7c9854247da9baefe43e573e9e2" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 45019 } }
I20260925 02:43:40.007717 22336 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9055c7c9854247da9baefe43e573e9e2 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:43:40.008527 22341 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9055c7c9854247da9baefe43e573e9e2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9055c7c9854247da9baefe43e573e9e2. Latest consensus state: current_term: 1 leader_uuid: "9055c7c9854247da9baefe43e573e9e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9055c7c9854247da9baefe43e573e9e2" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 45019 } } }
I20260925 02:43:40.009267 22341 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9055c7c9854247da9baefe43e573e9e2 [sys.catalog]: This master's current role is: LEADER
I20260925 02:43:40.009159 22343 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9055c7c9854247da9baefe43e573e9e2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9055c7c9854247da9baefe43e573e9e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9055c7c9854247da9baefe43e573e9e2" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 45019 } } }
I20260925 02:43:40.009593 22343 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9055c7c9854247da9baefe43e573e9e2 [sys.catalog]: This master's current role is: LEADER
W20260925 02:43:40.013104 22356 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9055c7c9854247da9baefe43e573e9e2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 02:43:40.013216 22356 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 02:43:40.013309 22354 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:43:40.014209 22354 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:43:40.020989 22354 catalog_manager.cc:1347] Generated new cluster ID: 0606dc2c5eb14faab0e302fda2c9fa86
I20260925 02:43:40.021093 22354 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 02:43:40.030606 22354 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 02:43:40.031659 22354 catalog_manager.cc:1504] Loading token signing keys...
I20260925 02:43:40.040078 22354 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9055c7c9854247da9baefe43e573e9e2: Generated new TSK 0
I20260925 02:43:40.040786 22354 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.157.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.157.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.157.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.129.keytab.
Entry for principal kudu/127.13.157.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.129.keytab.
I20260925 02:43:40.078534 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.157.129:0
--local_ip_for_outbound_sockets=127.13.157.129
--webserver_interface=127.13.157.129
--webserver_port=0
--tserver_master_addrs=127.13.157.190:45019
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.129.keytab
--principal=kudu/127.13.157.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:40879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:43:40.189013 22362 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:43:40.189260 22362 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:43:40.189302 22362 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:43:40.193251 22362 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:43:40.193305 22362 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:43:40.193362 22362 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.157.129
I20260925 02:43:40.198504 22362 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:40879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.129.keytab
--principal=kudu/127.13.157.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.157.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.157.129
--webserver_port=0
--tserver_master_addrs=127.13.157.190:45019
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22362
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.157.129
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:43:40.199548 22362 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:43:40.200610 22362 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:43:40.209986 22367 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:40.210058 22368 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:40.210031 22370 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:43:40.210131 22362 server_base.cc:1034] running on GCE node
I20260925 02:43:40.210717 22362 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:43:40.211398 22362 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:43:40.212567 22362 hybrid_clock.cc:648] HybridClock initialized: now 1790304220212547 us; error 33 us; skew 500 ppm
Sep 25 02:43:40 dist-test-slave-kznw krb5kdc[22255](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304220, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:43:40.215431 22362 init.cc:377] Logged in from keytab as kudu/127.13.157.129@KRBTEST.COM (short username kudu)
I20260925 02:43:40.216323 22362 webserver.cc:460] Webserver started at http://127.13.157.129:42307/ using document root <none> and password file <none>
I20260925 02:43:40.216866 22362 fs_manager.cc:359] Metadata directory not provided
I20260925 02:43:40.216928 22362 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:43:40.217127 22362 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:43:40.219326 22362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/ts-0/data/instance:
uuid: "dddb2e61280346a4aadf4770169165ac"
format_stamp: "Formatted at 2026-09-25 02:43:40 on dist-test-slave-kznw"
server_key: "9c721f70529a40fa7a8282a238ae81fa"
server_key_iv: "bd544cb2f972766f07be241aa1a413b1"
server_key_version: "encryptionkey@0"
I20260925 02:43:40.219929 22362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/ts-0/wal/instance:
uuid: "dddb2e61280346a4aadf4770169165ac"
format_stamp: "Formatted at 2026-09-25 02:43:40 on dist-test-slave-kznw"
server_key: "9c721f70529a40fa7a8282a238ae81fa"
server_key_iv: "bd544cb2f972766f07be241aa1a413b1"
server_key_version: "encryptionkey@0"
I20260925 02:43:40.223758 22362 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260925 02:43:40.226277 22377 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:40.227558 22362 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260925 02:43:40.227660 22362 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/ts-0/wal
uuid: "dddb2e61280346a4aadf4770169165ac"
format_stamp: "Formatted at 2026-09-25 02:43:40 on dist-test-slave-kznw"
server_key: "9c721f70529a40fa7a8282a238ae81fa"
server_key_iv: "bd544cb2f972766f07be241aa1a413b1"
server_key_version: "encryptionkey@0"
I20260925 02:43:40.227767 22362 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:43:40.254349 22362 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:43:40.257563 22362 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:43:40.257737 22362 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:43:40.258360 22362 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:43:40.259239 22362 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:43:40.259286 22362 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:40.259330 22362 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:43:40.259358 22362 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:40.267838 22362 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.129:34965
I20260925 02:43:40.267863 22491 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.129:34965 every 8 connection(s)
I20260925 02:43:40.268703 22362 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/ts-0/data/info.pb
I20260925 02:43:40.275221 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 22362
I20260925 02:43:40.275339 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/ts-0/wal/instance
I20260925 02:43:40.275638 13942 external_mini_cluster.cc:1428] Setting key b658355a78b06ad050a8a8881284abd0
Sep 25 02:43:40 dist-test-slave-kznw krb5kdc[22255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304220, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.129@KRBTEST.COM for kudu/127.13.157.190@KRBTEST.COM
I20260925 02:43:40.282212 22339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:40.270438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.129:46931 (local address 127.13.157.190:45019)
0925 02:43:40.270579 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:40.270583 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:40.271610 (+ 1027us) server_negotiation.cc:407] Connection header received
0925 02:43:40.272498 (+ 888us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:40.272501 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:40.272551 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:40.272649 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:40.273795 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:40.274396 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:40.275628 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:40.275953 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:40.276078 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:40.278768 (+ 2690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:40.278772 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:40.278775 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:43:40.278806 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:40.280148 (+ 1342us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:40.280705 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:40.280707 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:40.280708 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:40.280776 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:40.281123 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:40.281126 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:40.281128 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:40.281312 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:40.281389 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:40.281979 (+ 590us) server_negotiation.cc:299] Negotiation successful
0925 02:43:40.282080 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260925 02:43:40.283428 22494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:40.270819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:45019 (local address 127.13.157.129:46931)
0925 02:43:40.271463 (+ 644us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:40.271504 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:40.272286 (+ 782us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:40.272785 (+ 499us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:40.272792 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:40.273161 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:40.273633 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:40.273647 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:40.274534 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:40.274537 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:40.275510 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:40.275520 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:40.276160 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:40.276167 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:40.276345 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:40.276868 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:40.276884 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:40.278555 (+ 1671us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:43:40.280264 (+ 1709us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:40.280269 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:40.280284 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:40.280612 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:40.280851 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:40.280854 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:40.280856 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:40.281030 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:40.281409 (+ 379us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:43:40.281416 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:43:40.281740 (+ 324us) client_negotiation.cc:769] Sending connection context
0925 02:43:40.281962 (+ 222us) client_negotiation.cc:240] Negotiation successful
0925 02:43:40.282307 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":111,"threads_started":1}
I20260925 02:43:40.284711 22492 heartbeater.cc:347] Connected to a master server at 127.13.157.190:45019
I20260925 02:43:40.284929 22492 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.13.157.130@KRBTEST.COM; defaulting to no policy
I20260925 02:43:40.285415 22492 heartbeater.cc:511] Master 127.13.157.190:45019 requested a full tablet report, sending...
I20260925 02:43:40.286863 22300 ts_manager.cc:194] Registered new tserver with Master: dddb2e61280346a4aadf4770169165ac (127.13.157.129:34965)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.157.130@KRBTEST.COM" created.
I20260925 02:43:40.288231 22300 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.157.129@KRBTEST.COM'} at 127.13.157.129:46931
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.157.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.130.keytab.
Entry for principal kudu/127.13.157.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.130.keytab.
I20260925 02:43:40.302379 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.157.130:0
--local_ip_for_outbound_sockets=127.13.157.130
--webserver_interface=127.13.157.130
--webserver_port=0
--tserver_master_addrs=127.13.157.190:45019
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.130.keytab
--principal=kudu/127.13.157.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:40879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:43:40.411969 22497 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:43:40.412225 22497 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:43:40.412268 22497 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:43:40.416292 22497 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:43:40.416350 22497 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:43:40.416414 22497 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.157.130
I20260925 02:43:40.421770 22497 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:40879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.130.keytab
--principal=kudu/127.13.157.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.157.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.157.130
--webserver_port=0
--tserver_master_addrs=127.13.157.190:45019
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22497
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.157.130
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:43:40.422806 22497 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:43:40.423828 22497 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 02:43:40.433405 22497 server_base.cc:1034] running on GCE node
W20260925 02:43:40.433342 22503 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:40.433363 22502 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:40.433365 22505 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:43:40.433954 22497 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:43:40.434615 22497 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:43:40.435777 22497 hybrid_clock.cc:648] HybridClock initialized: now 1790304220435749 us; error 45 us; skew 500 ppm
Sep 25 02:43:40 dist-test-slave-kznw krb5kdc[22255](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304220, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:43:40.438423 22497 init.cc:377] Logged in from keytab as kudu/127.13.157.130@KRBTEST.COM (short username kudu)
I20260925 02:43:40.439327 22497 webserver.cc:460] Webserver started at http://127.13.157.130:41621/ using document root <none> and password file <none>
I20260925 02:43:40.439870 22497 fs_manager.cc:359] Metadata directory not provided
I20260925 02:43:40.439934 22497 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:43:40.440119 22497 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:43:40.442087 22497 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/ts-1/data/instance:
uuid: "01630e945ea44ae389c5e6ae98962df6"
format_stamp: "Formatted at 2026-09-25 02:43:40 on dist-test-slave-kznw"
server_key: "05a8ac7bef22a73597a3389e40cad163"
server_key_iv: "af389646899a395df8b9ce55c18e0e35"
server_key_version: "encryptionkey@0"
I20260925 02:43:40.442618 22497 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/ts-1/wal/instance:
uuid: "01630e945ea44ae389c5e6ae98962df6"
format_stamp: "Formatted at 2026-09-25 02:43:40 on dist-test-slave-kznw"
server_key: "05a8ac7bef22a73597a3389e40cad163"
server_key_iv: "af389646899a395df8b9ce55c18e0e35"
server_key_version: "encryptionkey@0"
I20260925 02:43:40.446187 22497 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260925 02:43:40.448702 22512 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:40.449858 22497 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260925 02:43:40.449962 22497 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/ts-1/wal
uuid: "01630e945ea44ae389c5e6ae98962df6"
format_stamp: "Formatted at 2026-09-25 02:43:40 on dist-test-slave-kznw"
server_key: "05a8ac7bef22a73597a3389e40cad163"
server_key_iv: "af389646899a395df8b9ce55c18e0e35"
server_key_version: "encryptionkey@0"
I20260925 02:43:40.450053 22497 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:43:40.474519 22497 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:43:40.477622 22497 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:43:40.477802 22497 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:43:40.478413 22497 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:43:40.479250 22497 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:43:40.479297 22497 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:40.479350 22497 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:43:40.479377 22497 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:40.487911 22497 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.130:44741
I20260925 02:43:40.487968 22626 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.130:44741 every 8 connection(s)
I20260925 02:43:40.488821 22497 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/ts-1/data/info.pb
Sep 25 02:43:40 dist-test-slave-kznw krb5kdc[22255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304220, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.130@KRBTEST.COM for kudu/127.13.157.190@KRBTEST.COM
I20260925 02:43:40.498759 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 22497
I20260925 02:43:40.498890 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/ts-1/wal/instance
I20260925 02:43:40.499289 13942 external_mini_cluster.cc:1428] Setting key 2f828651c5088d1fbd8912b46ae0fb49
I20260925 02:43:40.502290 22339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:40.490647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.130:41675 (local address 127.13.157.190:45019)
0925 02:43:40.490778 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:40.490782 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:40.491870 (+ 1088us) server_negotiation.cc:407] Connection header received
0925 02:43:40.492821 (+ 951us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:40.492825 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:40.492888 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:40.492971 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:40.494148 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:40.494687 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:40.495885 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:40.496121 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:40.496214 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:40.498691 (+ 2477us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:40.498694 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:40.498696 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:43:40.498722 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:40.500154 (+ 1432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:40.500675 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:40.500679 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:40.500680 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:40.500780 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:40.501207 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:40.501209 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:40.501211 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:40.501387 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:40.501461 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:40.502064 (+ 603us) server_negotiation.cc:299] Negotiation successful
0925 02:43:40.502170 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260925 02:43:40.503417 22629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:40.491042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:45019 (local address 127.13.157.130:41675)
0925 02:43:40.491709 (+ 667us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:40.491753 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:40.492609 (+ 856us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:40.493150 (+ 541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:40.493158 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:40.493517 (+ 359us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:40.493999 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:40.494014 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:40.494815 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:40.494817 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:40.495749 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:40.495760 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:40.496275 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:40.496279 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:40.496448 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:40.497034 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:40.497050 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:40.498526 (+ 1476us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:43:40.500261 (+ 1735us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:40.500267 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:40.500281 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:40.500565 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:40.500904 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:40.500908 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:40.500910 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:40.501113 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:40.501505 (+ 392us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:43:40.501512 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:43:40.501818 (+ 306us) client_negotiation.cc:769] Sending connection context
0925 02:43:40.502018 (+ 200us) client_negotiation.cc:240] Negotiation successful
0925 02:43:40.502345 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":110,"threads_started":1}
I20260925 02:43:40.504586 22627 heartbeater.cc:347] Connected to a master server at 127.13.157.190:45019
I20260925 02:43:40.504808 22627 heartbeater.cc:464] Registering TS with master...
I20260925 02:43:40.505306 22627 heartbeater.cc:511] Master 127.13.157.190:45019 requested a full tablet report, sending...
I20260925 02:43:40.506342 22300 ts_manager.cc:194] Registered new tserver with Master: 01630e945ea44ae389c5e6ae98962df6 (127.13.157.130:44741)
I20260925 02:43:40.506927 22300 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.157.130@KRBTEST.COM'} at 127.13.157.130:41675
WARNING: no policy specified for kudu/127.13.157.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.157.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.157.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.131.keytab.
Entry for principal kudu/127.13.157.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.131.keytab.
I20260925 02:43:40.526243 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.157.131:0
--local_ip_for_outbound_sockets=127.13.157.131
--webserver_interface=127.13.157.131
--webserver_port=0
--tserver_master_addrs=127.13.157.190:45019
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.131.keytab
--principal=kudu/127.13.157.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:40879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:43:40.635033 22632 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:43:40.635308 22632 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:43:40.635367 22632 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:43:40.639856 22632 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:43:40.639926 22632 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:43:40.640007 22632 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.157.131
I20260925 02:43:40.645484 22632 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:40879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.131.keytab
--principal=kudu/127.13.157.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.157.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.157.131
--webserver_port=0
--tserver_master_addrs=127.13.157.190:45019
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22632
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.157.131
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:43:40.646569 22632 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:43:40.647625 22632 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:43:40.657204 22638 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:40.657186 22637 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:40.657374 22640 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:43:40.657248 22632 server_base.cc:1034] running on GCE node
I20260925 02:43:40.657814 22632 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:43:40.658437 22632 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:43:40.659583 22632 hybrid_clock.cc:648] HybridClock initialized: now 1790304220659554 us; error 43 us; skew 500 ppm
Sep 25 02:43:40 dist-test-slave-kznw krb5kdc[22255](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304220, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:43:40.668538 22632 init.cc:377] Logged in from keytab as kudu/127.13.157.131@KRBTEST.COM (short username kudu)
I20260925 02:43:40.669423 22632 webserver.cc:460] Webserver started at http://127.13.157.131:40157/ using document root <none> and password file <none>
I20260925 02:43:40.669937 22632 fs_manager.cc:359] Metadata directory not provided
I20260925 02:43:40.669994 22632 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:43:40.670168 22632 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:43:40.672243 22632 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/ts-2/data/instance:
uuid: "08a2dbb7e7a04bfe90b5c11afa29ca41"
format_stamp: "Formatted at 2026-09-25 02:43:40 on dist-test-slave-kznw"
server_key: "99f5bd855b0186c61ea81da186ee119c"
server_key_iv: "d378389f003225bbbb45ee35dcaffebe"
server_key_version: "encryptionkey@0"
I20260925 02:43:40.672744 22632 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/ts-2/wal/instance:
uuid: "08a2dbb7e7a04bfe90b5c11afa29ca41"
format_stamp: "Formatted at 2026-09-25 02:43:40 on dist-test-slave-kznw"
server_key: "99f5bd855b0186c61ea81da186ee119c"
server_key_iv: "d378389f003225bbbb45ee35dcaffebe"
server_key_version: "encryptionkey@0"
I20260925 02:43:40.676379 22632 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260925 02:43:40.678802 22647 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:40.680121 22632 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260925 02:43:40.680222 22632 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/ts-2/wal
uuid: "08a2dbb7e7a04bfe90b5c11afa29ca41"
format_stamp: "Formatted at 2026-09-25 02:43:40 on dist-test-slave-kznw"
server_key: "99f5bd855b0186c61ea81da186ee119c"
server_key_iv: "d378389f003225bbbb45ee35dcaffebe"
server_key_version: "encryptionkey@0"
I20260925 02:43:40.680305 22632 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:43:40.695276 22632 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:43:40.698192 22632 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:43:40.698338 22632 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:43:40.698849 22632 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:43:40.699681 22632 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:43:40.699730 22632 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:40.699769 22632 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:43:40.699788 22632 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:40.708109 22632 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.131:34431
I20260925 02:43:40.708148 22761 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.131:34431 every 8 connection(s)
I20260925 02:43:40.709043 22632 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/ts-2/data/info.pb
I20260925 02:43:40.712405 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 22632
I20260925 02:43:40.712534 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/ts-2/wal/instance
I20260925 02:43:40.712791 13942 external_mini_cluster.cc:1428] Setting key b3df97af712bacec3482378bacc43bb6
Sep 25 02:43:40 dist-test-slave-kznw krb5kdc[22255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304220, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.131@KRBTEST.COM for kudu/127.13.157.190@KRBTEST.COM
I20260925 02:43:40.722532 22339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:40.710765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.131:45781 (local address 127.13.157.190:45019)
0925 02:43:40.710894 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:40.710897 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:40.711798 (+ 901us) server_negotiation.cc:407] Connection header received
0925 02:43:40.713398 (+ 1600us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:40.713402 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:40.713489 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:40.713583 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:40.714824 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:40.715436 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:40.716613 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:40.716831 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:40.716911 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:40.719336 (+ 2425us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:40.719340 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:40.719342 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:43:40.719368 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:40.720668 (+ 1300us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:40.721209 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:40.721211 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:40.721212 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:40.721284 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:40.721609 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:40.721611 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:40.721612 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:40.721772 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:40.721840 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:40.722309 (+ 469us) server_negotiation.cc:299] Negotiation successful
0925 02:43:40.722408 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260925 02:43:40.723388 22764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:40.711181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:45019 (local address 127.13.157.131:45781)
0925 02:43:40.711643 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:40.711694 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:40.712918 (+ 1224us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:40.713868 (+ 950us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:40.713875 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:40.714226 (+ 351us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:40.714707 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:40.714721 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:40.715578 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:40.715581 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:40.716474 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:40.716483 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:40.716959 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:40.716962 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:40.717113 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:40.717653 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:40.717669 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:40.719174 (+ 1505us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:43:40.720790 (+ 1616us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:40.720795 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:40.720807 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:40.721122 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:40.721384 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:40.721387 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:40.721388 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:40.721536 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:40.721841 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:43:40.721846 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:43:40.722115 (+ 269us) client_negotiation.cc:769] Sending connection context
0925 02:43:40.722269 (+ 154us) client_negotiation.cc:240] Negotiation successful
0925 02:43:40.722519 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":110,"threads_started":1}
I20260925 02:43:40.724504 22762 heartbeater.cc:347] Connected to a master server at 127.13.157.190:45019
I20260925 02:43:40.724764 22762 heartbeater.cc:464] Registering TS with master...
I20260925 02:43:40.725296 22762 heartbeater.cc:511] Master 127.13.157.190:45019 requested a full tablet report, sending...
I20260925 02:43:40.726342 22300 ts_manager.cc:194] Registered new tserver with Master: 08a2dbb7e7a04bfe90b5c11afa29ca41 (127.13.157.131:34431)
I20260925 02:43:40.726874 13942 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 02:43:40.727108 22300 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.157.131@KRBTEST.COM'} at 127.13.157.131:45781
WARNING: no policy specified for oryx/127.13.157.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.157.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.157.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/krb5kdc/oryx_127.13.157.129.keytab.
Entry for principal oryx/127.13.157.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/krb5kdc/oryx_127.13.157.129.keytab.
WARNING: no policy specified for oryx/127.13.157.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.157.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.157.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/krb5kdc/oryx_127.13.157.130.keytab.
Entry for principal oryx/127.13.157.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/krb5kdc/oryx_127.13.157.130.keytab.
WARNING: no policy specified for oryx/127.13.157.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.157.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.157.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/krb5kdc/oryx_127.13.157.131.keytab.
Entry for principal oryx/127.13.157.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/krb5kdc/oryx_127.13.157.131.keytab.
I20260925 02:43:40.797312 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 22362
I20260925 02:43:40.802613 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 22497
I20260925 02:43:40.807418 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 22632
I20260925 02:43:40.812319 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 22268
I20260925 02:43:40.817659 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.157.190:45019
--webserver_interface=127.13.157.190
--webserver_port=45455
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab
--principal=kudu/127.13.157.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:40879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.157.190:45019
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:43:40.928366 22771 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:43:40.928642 22771 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:43:40.928683 22771 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:43:40.932618 22771 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:43:40.932673 22771 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:43:40.932690 22771 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:43:40.932705 22771 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:43:40.932721 22771 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:43:40.938198 22771 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:40879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.157.190:45019
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab
--principal=kudu/127.13.157.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.157.190:45019
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.157.190
--webserver_port=45455
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22771
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:43:40.939302 22771 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:43:40.940303 22771 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:43:40.948513 22778 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:40.948494 22777 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:43:40.948662 22771 server_base.cc:1034] running on GCE node
W20260925 02:43:40.948812 22780 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:43:40.949352 22771 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:43:40.950487 22771 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:43:40.951659 22771 hybrid_clock.cc:648] HybridClock initialized: now 1790304220951637 us; error 31 us; skew 500 ppm
Sep 25 02:43:40 dist-test-slave-kznw krb5kdc[22255](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304220, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:43:40.954425 22771 init.cc:377] Logged in from keytab as kudu/127.13.157.190@KRBTEST.COM (short username kudu)
I20260925 02:43:40.955336 22771 webserver.cc:460] Webserver started at http://127.13.157.190:45455/ using document root <none> and password file <none>
I20260925 02:43:40.955852 22771 fs_manager.cc:359] Metadata directory not provided
I20260925 02:43:40.955931 22771 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:43:40.959827 22771 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260925 02:43:40.961983 22787 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:40.963279 22771 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260925 02:43:40.963397 22771 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/master-0/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/master-0/wal
uuid: "9055c7c9854247da9baefe43e573e9e2"
format_stamp: "Formatted at 2026-09-25 02:43:39 on dist-test-slave-kznw"
server_key: "1a82ca68cc8f49584cf9c6043689ff74"
server_key_iv: "93c886619de6080c709b0ffcbad2559a"
server_key_version: "encryptionkey@0"
I20260925 02:43:40.963923 22771 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:43:40.981007 22771 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:43:40.984195 22771 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:43:40.984378 22771 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:43:40.991200 22771 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.190:45019
I20260925 02:43:40.991231 22839 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.190:45019 every 8 connection(s)
I20260925 02:43:40.992399 22771 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
I20260925 02:43:40.993894 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 22771
I20260925 02:43:40.994445 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.157.129:34965
--local_ip_for_outbound_sockets=127.13.157.129
--tserver_master_addrs=127.13.157.190:45019
--webserver_port=42307
--webserver_interface=127.13.157.129
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.129.keytab
--principal=kudu/127.13.157.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:40879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.13.157.129
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/krb5kdc/oryx_127.13.157.129.keytab with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 02:43:40.997300 22840 sys_catalog.cc:263] Verifying existing consensus state
I20260925 02:43:40.998984 22840 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9055c7c9854247da9baefe43e573e9e2: Bootstrap starting.
I20260925 02:43:41.015570 22840 log.cc:824] T 00000000000000000000000000000000 P 9055c7c9854247da9baefe43e573e9e2: Log is configured to *not* fsync() on all Append() calls
I20260925 02:43:41.022279 22840 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9055c7c9854247da9baefe43e573e9e2: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 02:43:41.022641 22840 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9055c7c9854247da9baefe43e573e9e2: Bootstrap complete.
I20260925 02:43:41.026757 22840 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9055c7c9854247da9baefe43e573e9e2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9055c7c9854247da9baefe43e573e9e2" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 45019 } }
I20260925 02:43:41.027237 22840 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9055c7c9854247da9baefe43e573e9e2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9055c7c9854247da9baefe43e573e9e2, State: Initialized, Role: FOLLOWER
I20260925 02:43:41.027733 22840 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9055c7c9854247da9baefe43e573e9e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9055c7c9854247da9baefe43e573e9e2" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 45019 } }
I20260925 02:43:41.027848 22840 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9055c7c9854247da9baefe43e573e9e2 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 02:43:41.027907 22840 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9055c7c9854247da9baefe43e573e9e2 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 02:43:41.028003 22840 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9055c7c9854247da9baefe43e573e9e2 [term 1 FOLLOWER]: Advancing to term 2
I20260925 02:43:41.029999 22840 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9055c7c9854247da9baefe43e573e9e2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9055c7c9854247da9baefe43e573e9e2" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 45019 } }
I20260925 02:43:41.030323 22840 leader_election.cc:302] T 00000000000000000000000000000000 P 9055c7c9854247da9baefe43e573e9e2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9055c7c9854247da9baefe43e573e9e2; no voters:
I20260925 02:43:41.030622 22840 leader_election.cc:288] T 00000000000000000000000000000000 P 9055c7c9854247da9baefe43e573e9e2 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260925 02:43:41.030751 22845 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9055c7c9854247da9baefe43e573e9e2 [term 2 FOLLOWER]: Leader election won for term 2
I20260925 02:43:41.030997 22845 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9055c7c9854247da9baefe43e573e9e2 [term 2 LEADER]: Becoming Leader. State: Replica: 9055c7c9854247da9baefe43e573e9e2, State: Running, Role: LEADER
I20260925 02:43:41.031297 22845 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9055c7c9854247da9baefe43e573e9e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9055c7c9854247da9baefe43e573e9e2" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 45019 } }
I20260925 02:43:41.031929 22840 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9055c7c9854247da9baefe43e573e9e2 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:43:41.032755 22847 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9055c7c9854247da9baefe43e573e9e2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9055c7c9854247da9baefe43e573e9e2. Latest consensus state: current_term: 2 leader_uuid: "9055c7c9854247da9baefe43e573e9e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9055c7c9854247da9baefe43e573e9e2" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 45019 } } }
I20260925 02:43:41.032874 22847 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9055c7c9854247da9baefe43e573e9e2 [sys.catalog]: This master's current role is: LEADER
I20260925 02:43:41.033215 22854 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:43:41.033221 22846 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9055c7c9854247da9baefe43e573e9e2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "9055c7c9854247da9baefe43e573e9e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9055c7c9854247da9baefe43e573e9e2" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 45019 } } }
I20260925 02:43:41.033339 22846 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9055c7c9854247da9baefe43e573e9e2 [sys.catalog]: This master's current role is: LEADER
I20260925 02:43:41.036656 22854 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:43:41.037307 22854 catalog_manager.cc:1259] Loaded cluster ID: 0606dc2c5eb14faab0e302fda2c9fa86
I20260925 02:43:41.037361 22854 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 02:43:41.039326 22854 catalog_manager.cc:1504] Loading token signing keys...
I20260925 02:43:41.039906 22854 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 9055c7c9854247da9baefe43e573e9e2: Loaded TSK: 0
I20260925 02:43:41.040824 22854 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260925 02:43:41.116039 22842 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:43:41.116299 22842 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:43:41.116343 22842 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:43:41.120376 22842 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:43:41.120433 22842 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:43:41.120498 22842 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.157.129
I20260925 02:43:41.125816 22842 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:40879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/krb5kdc/oryx_127.13.157.129.keytab
--principal=oryx/127.13.157.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.157.129:34965
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.157.129
--webserver_port=42307
--tserver_master_addrs=127.13.157.190:45019
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22842
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.157.129
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:43:41.126869 22842 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:43:41.127929 22842 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:43:41.138207 22868 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:41.138185 22867 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:41.138190 22870 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:43:41.138509 22842 server_base.cc:1034] running on GCE node
I20260925 02:43:41.139040 22842 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:43:41.139683 22842 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:43:41.140874 22842 hybrid_clock.cc:648] HybridClock initialized: now 1790304221140831 us; error 58 us; skew 500 ppm
Sep 25 02:43:41 dist-test-slave-kznw krb5kdc[22255](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304221, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.157.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:43:41.143625 22842 init.cc:377] Logged in from keytab as oryx/127.13.157.129@KRBTEST.COM (short username oryx)
I20260925 02:43:41.144506 22842 webserver.cc:460] Webserver started at http://127.13.157.129:42307/ using document root <none> and password file <none>
I20260925 02:43:41.145187 22842 fs_manager.cc:359] Metadata directory not provided
I20260925 02:43:41.145272 22842 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:43:41.149324 22842 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.002s
I20260925 02:43:41.151597 22877 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:41.153028 22842 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 02:43:41.153178 22842 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/ts-0/wal
uuid: "dddb2e61280346a4aadf4770169165ac"
format_stamp: "Formatted at 2026-09-25 02:43:40 on dist-test-slave-kznw"
server_key: "9c721f70529a40fa7a8282a238ae81fa"
server_key_iv: "bd544cb2f972766f07be241aa1a413b1"
server_key_version: "encryptionkey@0"
I20260925 02:43:41.153715 22842 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:43:41.170207 22842 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:43:41.173408 22842 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:43:41.173588 22842 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:43:41.174189 22842 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:43:41.175077 22842 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:43:41.175125 22842 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:41.175184 22842 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:43:41.175212 22842 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:41.184151 22842 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.129:34965
I20260925 02:43:41.184221 22991 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.129:34965 every 8 connection(s)
I20260925 02:43:41.185148 22842 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/ts-0/data/info.pb
I20260925 02:43:41.191380 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 22842
I20260925 02:43:41.191865 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.157.130:44741
--local_ip_for_outbound_sockets=127.13.157.130
--tserver_master_addrs=127.13.157.190:45019
--webserver_port=41621
--webserver_interface=127.13.157.130
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.130.keytab
--principal=kudu/127.13.157.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:40879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.13.157.130
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/krb5kdc/oryx_127.13.157.130.keytab with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 25 02:43:41 dist-test-slave-kznw krb5kdc[22255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.157.129@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 02:43:41 dist-test-slave-kznw krb5kdc[22255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.157.129@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 02:43:41.198437 22995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:41.187375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:45019 (local address 127.13.157.129:60627)
0925 02:43:41.188040 (+ 665us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:41.188103 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:41.188871 (+ 768us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:41.189909 (+ 1038us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:41.189915 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:41.190280 (+ 365us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:41.190759 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:41.190774 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:41.192429 (+ 1655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:41.192432 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:41.193617 (+ 1185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:41.193627 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:41.194615 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:41.194618 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:41.194754 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:41.195302 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:41.195318 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:41.197469 (+ 2151us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.157.190:45019: Server oryx/127.13.157.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":109,"threads_started":1}
W20260925 02:43:41.198429 22994 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:43:41.187322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.129:60627 (local address 127.13.157.190:45019)
0925 02:43:41.187759 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:41.187775 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:41.188221 (+ 446us) server_negotiation.cc:407] Connection header received
0925 02:43:41.189223 (+ 1002us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:41.189234 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:41.189538 (+ 304us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:41.189816 (+ 278us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:41.190943 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:41.192286 (+ 1343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:41.193732 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:41.194446 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:41.194582 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:41.197711 (+ 3129us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.157.129:60627: BlockingRecv error: recv got EOF from 127.13.157.129:60627 (error 108)
Metrics: {"server-negotiator.queue_time_us":281,"thread_start_us":104,"threads_started":1}
W20260925 02:43:41.199079 22992 heartbeater.cc:644] Failed to heartbeat to 127.13.157.190:45019 (0 consecutive failures): Not authorized: Failed to ping master at 127.13.157.190:45019: Client connection negotiation failed: client connection to 127.13.157.190:45019: Server oryx/127.13.157.190@KRBTEST.COM not found in Kerberos database
Sep 25 02:43:41 dist-test-slave-kznw krb5kdc[22255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.157.129@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 02:43:41 dist-test-slave-kznw krb5kdc[22255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.157.129@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 02:43:41.204632 22995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:41.199455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:45019 (local address 127.13.157.129:44733)
0925 02:43:41.199644 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:41.199669 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:41.199766 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:41.199988 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:41.199992 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:41.200154 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:41.200273 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:41.200281 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:41.200973 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:41.200977 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:41.201837 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:41.201849 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:41.202252 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:41.202257 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:41.202349 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:41.202772 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:41.202785 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:41.204505 (+ 1720us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.157.190:45019: Server oryx/127.13.157.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":113,"mutex_wait_us":62}
W20260925 02:43:41.204747 22994 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:43:41.199533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.129:44733 (local address 127.13.157.190:45019)
0925 02:43:41.199649 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:41.199653 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:41.199716 (+ 63us) server_negotiation.cc:407] Connection header received
0925 02:43:41.199845 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:41.199847 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:41.199887 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:41.199948 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:41.200369 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:41.200874 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:41.201943 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:41.202153 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:41.202224 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:41.204640 (+ 2416us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.157.129:44733: BlockingRecv error: recv got EOF from 127.13.157.129:44733 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Sep 25 02:43:41 dist-test-slave-kznw krb5kdc[22255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.157.129@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 02:43:41 dist-test-slave-kznw krb5kdc[22255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.157.129@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 02:43:41.210114 22995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:41.205266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:45019 (local address 127.13.157.129:51273)
0925 02:43:41.205417 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:41.205430 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:41.205491 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:41.205676 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:41.205678 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:41.205790 (+ 112us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:41.205853 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:41.205857 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:41.206821 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:41.206823 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:41.207389 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:41.207398 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:41.207909 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:41.207911 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:41.207966 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:41.208412 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:41.208422 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:41.209992 (+ 1570us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.157.190:45019: Server oryx/127.13.157.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":92}
W20260925 02:43:41.210206 22994 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:43:41.205369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.129:51273 (local address 127.13.157.190:45019)
0925 02:43:41.205488 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:41.205492 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:41.205506 (+ 14us) server_negotiation.cc:407] Connection header received
0925 02:43:41.205541 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:41.205544 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:41.205585 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:41.205661 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:41.205937 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:41.206709 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:41.207485 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:41.207819 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:41.207912 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:41.210107 (+ 2195us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.157.129:51273: BlockingRecv error: recv got EOF from 127.13.157.129:51273 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260925 02:43:41.210394 22992 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260925 02:43:41.306267 22996 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:43:41.306563 22996 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:43:41.306625 22996 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:43:41.310667 22996 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:43:41.310737 22996 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:43:41.310822 22996 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.157.130
I20260925 02:43:41.316192 22996 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:40879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/krb5kdc/oryx_127.13.157.130.keytab
--principal=oryx/127.13.157.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.157.130:44741
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.157.130
--webserver_port=41621
--tserver_master_addrs=127.13.157.190:45019
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22996
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.157.130
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:43:41.317301 22996 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:43:41.318346 22996 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:43:41.328076 23004 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:41.328092 23001 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:41.328092 23002 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:43:41.328323 22996 server_base.cc:1034] running on GCE node
I20260925 02:43:41.328745 22996 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:43:41.329391 22996 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:43:41.330559 22996 hybrid_clock.cc:648] HybridClock initialized: now 1790304221330527 us; error 42 us; skew 500 ppm
Sep 25 02:43:41 dist-test-slave-kznw krb5kdc[22255](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304221, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.157.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:43:41.333585 22996 init.cc:377] Logged in from keytab as oryx/127.13.157.130@KRBTEST.COM (short username oryx)
I20260925 02:43:41.334467 22996 webserver.cc:460] Webserver started at http://127.13.157.130:41621/ using document root <none> and password file <none>
I20260925 02:43:41.335068 22996 fs_manager.cc:359] Metadata directory not provided
I20260925 02:43:41.335155 22996 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:43:41.339358 22996 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260925 02:43:41.341573 23011 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:41.342869 22996 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260925 02:43:41.343047 22996 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/ts-1/wal
uuid: "01630e945ea44ae389c5e6ae98962df6"
format_stamp: "Formatted at 2026-09-25 02:43:40 on dist-test-slave-kznw"
server_key: "05a8ac7bef22a73597a3389e40cad163"
server_key_iv: "af389646899a395df8b9ce55c18e0e35"
server_key_version: "encryptionkey@0"
I20260925 02:43:41.343585 22996 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:43:41.361323 22996 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:43:41.364336 22996 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:43:41.364509 22996 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:43:41.365085 22996 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:43:41.365872 22996 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:43:41.365917 22996 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:41.365967 22996 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:43:41.365994 22996 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:41.374423 22996 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.130:44741
I20260925 02:43:41.374462 23125 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.130:44741 every 8 connection(s)
I20260925 02:43:41.375387 22996 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/ts-1/data/info.pb
I20260925 02:43:41.378988 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 22996
I20260925 02:43:41.379410 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.157.131:34431
--local_ip_for_outbound_sockets=127.13.157.131
--tserver_master_addrs=127.13.157.190:45019
--webserver_port=40157
--webserver_interface=127.13.157.131
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.131.keytab
--principal=kudu/127.13.157.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:40879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.13.157.131
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/krb5kdc/oryx_127.13.157.131.keytab with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 25 02:43:41 dist-test-slave-kznw krb5kdc[22255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.157.130@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 02:43:41 dist-test-slave-kznw krb5kdc[22255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.157.130@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM, Server not found in Kerberos database
W20260925 02:43:41.386320 22994 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:43:41.377218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.130:53565 (local address 127.13.157.190:45019)
0925 02:43:41.377384 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:41.377388 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:41.378395 (+ 1007us) server_negotiation.cc:407] Connection header received
0925 02:43:41.379399 (+ 1004us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:41.379403 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:41.379464 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:41.379536 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:41.380801 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:41.381330 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:41.382451 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:41.382734 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:41.382828 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:41.386187 (+ 3359us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.157.130:53565: BlockingRecv error: recv got EOF from 127.13.157.130:53565 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
I20260925 02:43:41.387049 23128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:41.377593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:45019 (local address 127.13.157.130:53565)
0925 02:43:41.378243 (+ 650us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:41.378284 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:41.379197 (+ 913us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:41.379819 (+ 622us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:41.379826 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:41.380177 (+ 351us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:41.380653 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:41.380667 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:41.381451 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:41.381454 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:41.382331 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:41.382341 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:41.382881 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:41.382885 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:41.383084 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:41.383617 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:41.383633 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:41.386033 (+ 2400us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.157.190:45019: Server oryx/127.13.157.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":115,"threads_started":1}
W20260925 02:43:41.387493 23126 heartbeater.cc:644] Failed to heartbeat to 127.13.157.190:45019 (0 consecutive failures): Not authorized: Failed to ping master at 127.13.157.190:45019: Client connection negotiation failed: client connection to 127.13.157.190:45019: Server oryx/127.13.157.190@KRBTEST.COM not found in Kerberos database
Sep 25 02:43:41 dist-test-slave-kznw krb5kdc[22255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.157.130@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 02:43:41 dist-test-slave-kznw krb5kdc[22255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.157.130@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 02:43:41.392700 23128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:41.387927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:45019 (local address 127.13.157.130:44145)
0925 02:43:41.388046 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:41.388061 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:41.388140 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:41.388360 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:41.388362 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:41.388497 (+ 135us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:41.388610 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:41.388616 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:41.389309 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:41.389312 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:41.389871 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:41.389878 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:41.390287 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:41.390291 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:41.390380 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:41.390796 (+ 416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:41.390808 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:41.392577 (+ 1769us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.157.190:45019: Server oryx/127.13.157.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":43}
W20260925 02:43:41.392966 22994 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:43:41.388011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.130:44145 (local address 127.13.157.190:45019)
0925 02:43:41.388173 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:41.388177 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:41.388191 (+ 14us) server_negotiation.cc:407] Connection header received
0925 02:43:41.388231 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:41.388233 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:41.388270 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:41.388331 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:41.388719 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:41.389214 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:41.389977 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:41.390192 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:41.390275 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:41.392745 (+ 2470us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.157.130:44145: BlockingRecv error: recv got EOF from 127.13.157.130:44145 (error 108)
Metrics: {"server-negotiator.queue_time_us":97}
Sep 25 02:43:41 dist-test-slave-kznw krb5kdc[22255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.157.130@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 02:43:41 dist-test-slave-kznw krb5kdc[22255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.157.130@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 02:43:41.397794 23128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:41.393278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:45019 (local address 127.13.157.130:51477)
0925 02:43:41.393378 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:41.393393 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:41.393470 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:41.393707 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:41.393709 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:41.393841 (+ 132us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:41.393910 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:41.393914 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:41.394597 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:41.394600 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:41.395161 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:41.395170 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:41.395567 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:41.395571 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:41.395643 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:41.396083 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:41.396094 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:41.397682 (+ 1588us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.157.190:45019: Server oryx/127.13.157.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":29}
W20260925 02:43:41.397904 22994 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:43:41.393341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.130:51477 (local address 127.13.157.190:45019)
0925 02:43:41.393435 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:41.393439 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:41.393451 (+ 12us) server_negotiation.cc:407] Connection header received
0925 02:43:41.393577 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:41.393580 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:41.393620 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:41.393679 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:41.394014 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:41.394495 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:41.395265 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:41.395474 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:41.395567 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:41.397807 (+ 2240us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.157.130:51477: BlockingRecv error: recv got EOF from 127.13.157.130:51477 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
W20260925 02:43:41.398136 23126 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260925 02:43:41.490471 23129 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:43:41.490728 23129 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:43:41.490768 23129 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:43:41.494717 23129 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:43:41.494771 23129 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:43:41.494825 23129 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.157.131
I20260925 02:43:41.500010 23129 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:40879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/krb5kdc/oryx_127.13.157.131.keytab
--principal=oryx/127.13.157.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.157.131:34431
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.157.131
--webserver_port=40157
--tserver_master_addrs=127.13.157.190:45019
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23129
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.157.131
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:43:41.500942 23129 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:43:41.501881 23129 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:43:41.511638 23134 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:41.511638 23137 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:41.511646 23135 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:43:41.511883 23129 server_base.cc:1034] running on GCE node
I20260925 02:43:41.512267 23129 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:43:41.512880 23129 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:43:41.514110 23129 hybrid_clock.cc:648] HybridClock initialized: now 1790304221514099 us; error 27 us; skew 500 ppm
Sep 25 02:43:41 dist-test-slave-kznw krb5kdc[22255](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304221, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.157.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:43:41.516707 23129 init.cc:377] Logged in from keytab as oryx/127.13.157.131@KRBTEST.COM (short username oryx)
I20260925 02:43:41.517586 23129 webserver.cc:460] Webserver started at http://127.13.157.131:40157/ using document root <none> and password file <none>
I20260925 02:43:41.518118 23129 fs_manager.cc:359] Metadata directory not provided
I20260925 02:43:41.518196 23129 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:43:41.522121 23129 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260925 02:43:41.524211 23144 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:41.525530 23129 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260925 02:43:41.525655 23129 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/ts-2/wal
uuid: "08a2dbb7e7a04bfe90b5c11afa29ca41"
format_stamp: "Formatted at 2026-09-25 02:43:40 on dist-test-slave-kznw"
server_key: "99f5bd855b0186c61ea81da186ee119c"
server_key_iv: "d378389f003225bbbb45ee35dcaffebe"
server_key_version: "encryptionkey@0"
I20260925 02:43:41.526194 23129 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:43:41.544034 23129 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:43:41.547171 23129 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:43:41.547416 23129 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:43:41.548012 23129 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:43:41.548822 23129 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:43:41.548867 23129 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:41.548921 23129 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:43:41.548946 23129 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:41.557271 23129 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.131:34431
I20260925 02:43:41.557329 23258 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.131:34431 every 8 connection(s)
I20260925 02:43:41.558279 23129 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790304178325258-13942-0/minicluster-data/ts-2/data/info.pb
I20260925 02:43:41.565563 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 23129
Sep 25 02:43:41 dist-test-slave-kznw krb5kdc[22255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.157.131@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 02:43:41 dist-test-slave-kznw krb5kdc[22255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.157.131@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM, Server not found in Kerberos database
W20260925 02:43:41.568724 22994 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:43:41.560027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.131:54859 (local address 127.13.157.190:45019)
0925 02:43:41.560168 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:41.560171 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:41.561183 (+ 1012us) server_negotiation.cc:407] Connection header received
0925 02:43:41.562024 (+ 841us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:41.562028 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:41.562074 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:41.562142 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:41.563291 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:41.563812 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:41.564925 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:41.565150 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:41.565229 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:41.568614 (+ 3385us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.157.131:54859: BlockingRecv error: recv got EOF from 127.13.157.131:54859 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
I20260925 02:43:41.569412 23261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:41.560446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:45019 (local address 127.13.157.131:54859)
0925 02:43:41.561059 (+ 613us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:41.561097 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:41.561834 (+ 737us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:41.562279 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:41.562286 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:41.562622 (+ 336us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:41.563151 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:41.563165 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:41.563913 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:41.563916 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:41.564813 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:41.564822 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:41.565267 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:41.565269 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:41.565405 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:41.565969 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:41.565989 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:41.568529 (+ 2540us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.157.190:45019: Server oryx/127.13.157.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":107,"threads_started":1}
W20260925 02:43:41.569921 23259 heartbeater.cc:644] Failed to heartbeat to 127.13.157.190:45019 (0 consecutive failures): Not authorized: Failed to ping master at 127.13.157.190:45019: Client connection negotiation failed: client connection to 127.13.157.190:45019: Server oryx/127.13.157.190@KRBTEST.COM not found in Kerberos database
Sep 25 02:43:41 dist-test-slave-kznw krb5kdc[22255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.157.131@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 02:43:41 dist-test-slave-kznw krb5kdc[22255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.157.131@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 02:43:41.575889 23261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:41.570353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:45019 (local address 127.13.157.131:37943)
0925 02:43:41.570499 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:41.570512 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:41.570572 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:41.570817 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:41.570819 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:41.570963 (+ 144us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:41.571079 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:41.571085 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:41.572072 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:41.572074 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:41.572774 (+ 700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:41.572786 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:41.573307 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:41.573311 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:41.573400 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:41.573673 (+ 273us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:41.573686 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:41.575779 (+ 2093us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.157.190:45019: Server oryx/127.13.157.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":90}
W20260925 02:43:41.576109 22994 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:43:41.570446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.131:37943 (local address 127.13.157.190:45019)
0925 02:43:41.570555 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:41.570558 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:41.570571 (+ 13us) server_negotiation.cc:407] Connection header received
0925 02:43:41.570667 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:41.570671 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:41.570716 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:41.570811 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:41.571192 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:41.571933 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:41.572882 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:41.573198 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:41.573286 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:41.575945 (+ 2659us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.157.131:37943: BlockingRecv error: recv got EOF from 127.13.157.131:37943 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
I20260925 02:43:41.576853 23263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:41.566499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47840 (local address 127.13.157.190:45019)
0925 02:43:41.566763 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:41.566767 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:41.566779 (+ 12us) server_negotiation.cc:407] Connection header received
0925 02:43:41.567245 (+ 466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:41.567249 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:41.567302 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:41.567380 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:41.568268 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:41.569019 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:41.570146 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:41.570460 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:41.570564 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:41.572398 (+ 1834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:41.572405 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:41.572423 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:43:41.572453 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:41.574414 (+ 1961us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:41.574810 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:41.574816 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:41.574822 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:41.574996 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:41.575286 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:41.575289 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:41.575290 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:41.575709 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:41.575889 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:41.576491 (+ 602us) server_negotiation.cc:299] Negotiation successful
0925 02:43:41.576742 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":190,"thread_start_us":87,"threads_started":1}
Sep 25 02:43:41 dist-test-slave-kznw krb5kdc[22255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.157.131@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 02:43:41 dist-test-slave-kznw krb5kdc[22255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.157.131@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 02:43:41.581380 23264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:41.576389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:45019 (local address 127.13.157.131:38519)
0925 02:43:41.576700 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:41.576711 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:41.576800 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:41.577076 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:41.577078 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:41.577221 (+ 143us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:41.577304 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:41.577309 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:41.578123 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:41.578126 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:41.578861 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:41.578872 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:41.579364 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:41.579367 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:41.579445 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:41.579839 (+ 394us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:41.579850 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:41.581267 (+ 1417us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.157.190:45019: Server oryx/127.13.157.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":101,"threads_started":1}
W20260925 02:43:41.581501 22994 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:43:41.576448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.131:38519 (local address 127.13.157.190:45019)
0925 02:43:41.576610 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:41.576614 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:41.576775 (+ 161us) server_negotiation.cc:407] Connection header received
0925 02:43:41.576866 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:41.576869 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:41.576907 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:41.576983 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:41.577403 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:41.578020 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:41.579005 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:41.579254 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:41.579349 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:41.581381 (+ 2032us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.157.131:38519: BlockingRecv error: recv got EOF from 127.13.157.131:38519 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
W20260925 02:43:41.581543 23259 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Sep 25 02:43:42 dist-test-slave-kznw krb5kdc[22255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.157.129@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 02:43:42 dist-test-slave-kznw krb5kdc[22255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.157.129@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 02:43:42.216396 23265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:42.210962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:45019 (local address 127.13.157.129:36791)
0925 02:43:42.211259 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:42.211274 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:42.211378 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:42.211631 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:42.211633 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:42.211821 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:42.211946 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:42.211954 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:42.212721 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:42.212724 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:42.213305 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:42.213314 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:42.213763 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:42.213767 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:42.213866 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:42.214343 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:42.214358 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:42.216278 (+ 1920us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.157.190:45019: Server oryx/127.13.157.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":72,"threads_started":1}
W20260925 02:43:42.216620 23266 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:43:42.211088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.129:36791 (local address 127.13.157.190:45019)
0925 02:43:42.211321 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:42.211325 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:42.211338 (+ 13us) server_negotiation.cc:407] Connection header received
0925 02:43:42.211466 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:42.211469 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:42.211516 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:42.211597 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:42.212064 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:42.212614 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:42.213412 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:42.213658 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:42.213745 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:42.216449 (+ 2704us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.157.129:36791: BlockingRecv error: recv got EOF from 127.13.157.129:36791 (error 108)
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":82,"threads_started":1}
Sep 25 02:43:42 dist-test-slave-kznw krb5kdc[22255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.157.130@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 02:43:42 dist-test-slave-kznw krb5kdc[22255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.157.130@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 02:43:42.404127 23267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:42.398812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:45019 (local address 127.13.157.130:35687)
0925 02:43:42.399096 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:42.399113 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:42.399225 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:42.399477 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:42.399480 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:42.399693 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:42.399816 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:42.399823 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:42.400672 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:42.400674 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:42.401298 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:42.401308 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:42.401722 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:42.401725 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:42.401786 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:42.402241 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:42.402252 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:42.403992 (+ 1740us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.157.190:45019: Server oryx/127.13.157.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":108,"threads_started":1}
W20260925 02:43:42.404278 23266 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:43:42.398952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.130:35687 (local address 127.13.157.190:45019)
0925 02:43:42.399213 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:42.399217 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:42.399231 (+ 14us) server_negotiation.cc:407] Connection header received
0925 02:43:42.399308 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:42.399311 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:42.399357 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:42.399428 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:42.399918 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:42.400556 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:42.401412 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:42.401634 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:42.401712 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:42.404165 (+ 2453us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.157.130:35687: BlockingRecv error: recv got EOF from 127.13.157.130:35687 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Sep 25 02:43:42 dist-test-slave-kznw krb5kdc[22255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.157.131@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 02:43:42 dist-test-slave-kznw krb5kdc[22255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.157.131@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 02:43:42.587716 23268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:42.582156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:45019 (local address 127.13.157.131:33249)
0925 02:43:42.582439 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:42.582455 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:42.582585 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:42.582835 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:42.582837 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:42.583062 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:42.583181 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:42.583187 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:42.583945 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:42.583949 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:42.584537 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:42.584544 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:42.584989 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:42.584993 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:42.585077 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:42.585583 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:42.585594 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:42.587583 (+ 1989us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.157.190:45019: Server oryx/127.13.157.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":85,"threads_started":1}
W20260925 02:43:42.587833 23266 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:43:42.582271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.131:33249 (local address 127.13.157.190:45019)
0925 02:43:42.582439 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:42.582444 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:42.582530 (+ 86us) server_negotiation.cc:407] Connection header received
0925 02:43:42.582678 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:42.582681 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:42.582731 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:42.582797 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:42.583304 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:42.583837 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:42.584651 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:42.584868 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:42.584953 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:42.587714 (+ 2761us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.157.131:33249: BlockingRecv error: recv got EOF from 127.13.157.131:33249 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Sep 25 02:43:43 dist-test-slave-kznw krb5kdc[22255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.157.129@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 02:43:43 dist-test-slave-kznw krb5kdc[22255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.157.129@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 02:43:43.222972 23269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:43.217227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:45019 (local address 127.13.157.129:39705)
0925 02:43:43.217532 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:43.217547 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:43.217643 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:43.217934 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:43.217938 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:43.218116 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:43.218220 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:43.218226 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:43.219065 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:43.219069 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:43.219668 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:43.219678 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:43.220131 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:43.220134 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:43.220199 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:43.220694 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:43.220706 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:43.222793 (+ 2087us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.157.190:45019: Server oryx/127.13.157.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":73,"threads_started":1}
W20260925 02:43:43.223145 23270 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:43:43.217316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.129:39705 (local address 127.13.157.190:45019)
0925 02:43:43.217626 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:43.217630 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:43.217643 (+ 13us) server_negotiation.cc:407] Connection header received
0925 02:43:43.217736 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:43.217738 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:43.217783 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:43.217862 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:43.218361 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:43.218958 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:43.219792 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:43.220025 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:43.220111 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:43.223010 (+ 2899us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.157.129:39705: BlockingRecv error: recv got EOF from 127.13.157.129:39705 (error 108)
Metrics: {"server-negotiator.queue_time_us":218,"thread_start_us":110,"threads_started":1}
Sep 25 02:43:43 dist-test-slave-kznw krb5kdc[22255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.157.130@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 02:43:43 dist-test-slave-kznw krb5kdc[22255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.157.130@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 02:43:43.409958 23271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:43.404864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:45019 (local address 127.13.157.130:34633)
0925 02:43:43.405165 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:43.405178 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:43.405279 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:43.405490 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:43.405492 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:43.405661 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:43.405761 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:43.405766 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:43.406565 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:43.406569 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:43.407194 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:43.407204 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:43.407603 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:43.407605 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:43.407664 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:43.408126 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:43.408136 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:43.409822 (+ 1686us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.157.190:45019: Server oryx/127.13.157.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":70,"threads_started":1}
W20260925 02:43:43.410058 23270 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:43:43.405043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.130:34633 (local address 127.13.157.190:45019)
0925 02:43:43.405173 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:43.405176 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:43.405272 (+ 96us) server_negotiation.cc:407] Connection header received
0925 02:43:43.405345 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:43.405347 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:43.405395 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:43.405463 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:43.405884 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:43.406437 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:43.407287 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:43.407520 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:43.407599 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:43.409955 (+ 2356us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.157.130:34633: BlockingRecv error: recv got EOF from 127.13.157.130:34633 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Sep 25 02:43:43 dist-test-slave-kznw krb5kdc[22255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.157.131@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 02:43:43 dist-test-slave-kznw krb5kdc[22255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.157.131@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 02:43:43.593752 23272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:43.588414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:45019 (local address 127.13.157.131:51859)
0925 02:43:43.588681 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:43.588694 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:43.588789 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:43.589073 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:43.589075 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:43.589252 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:43.589348 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:43.589354 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:43.590172 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:43.590179 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:43.590754 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:43.590763 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:43.591305 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:43.591308 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:43.591373 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:43.591871 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:43.591884 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:43.593637 (+ 1753us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.157.190:45019: Server oryx/127.13.157.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":72,"threads_started":1}
W20260925 02:43:43.593876 23270 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:43:43.588581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.131:51859 (local address 127.13.157.190:45019)
0925 02:43:43.588697 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:43.588701 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:43.588742 (+ 41us) server_negotiation.cc:407] Connection header received
0925 02:43:43.588868 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:43.588870 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:43.588968 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:43.589041 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:43.589457 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:43.590041 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:43.590877 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:43.591201 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:43.591296 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:43.593768 (+ 2472us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.157.131:51859: BlockingRecv error: recv got EOF from 127.13.157.131:51859 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Sep 25 02:43:44 dist-test-slave-kznw krb5kdc[22255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.157.129@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 02:43:44 dist-test-slave-kznw krb5kdc[22255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.157.129@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 02:43:44.229624 23273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:44.223820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:45019 (local address 127.13.157.129:60087)
0925 02:43:44.224150 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:44.224164 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:44.224266 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:44.224551 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:44.224555 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:44.224732 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:44.224834 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:44.224840 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:44.225612 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:44.225616 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:44.226242 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:44.226252 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:44.226697 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:44.226700 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:44.226769 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:44.227252 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:44.227263 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:44.229434 (+ 2171us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.157.190:45019: Server oryx/127.13.157.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":75,"threads_started":1}
W20260925 02:43:44.229686 23274 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:43:44.223911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.129:60087 (local address 127.13.157.190:45019)
0925 02:43:44.224203 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:44.224207 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:44.224220 (+ 13us) server_negotiation.cc:407] Connection header received
0925 02:43:44.224362 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:44.224364 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:44.224408 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:44.224481 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:44.224950 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:44.225510 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:44.226362 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:44.226596 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:44.226673 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:44.229557 (+ 2884us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.157.129:60087: BlockingRecv error: recv got EOF from 127.13.157.129:60087 (error 108)
Metrics: {"server-negotiator.queue_time_us":217,"thread_start_us":111,"threads_started":1}
Sep 25 02:43:44 dist-test-slave-kznw krb5kdc[22255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.157.130@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 02:43:44 dist-test-slave-kznw krb5kdc[22255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.157.130@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 02:43:44.416356 23275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:44.410752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:45019 (local address 127.13.157.130:35455)
0925 02:43:44.411030 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:44.411047 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:44.411154 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:44.411419 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:44.411422 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:44.411614 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:44.411722 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:44.411728 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:44.412500 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:44.412503 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:44.413075 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:44.413083 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:44.413502 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:44.413505 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:44.413565 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:44.414117 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:44.414128 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:44.416189 (+ 2061us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.157.190:45019: Server oryx/127.13.157.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":75,"threads_started":1}
W20260925 02:43:44.416476 23274 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:43:44.410827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.130:35455 (local address 127.13.157.190:45019)
0925 02:43:44.411002 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:44.411007 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:44.411138 (+ 131us) server_negotiation.cc:407] Connection header received
0925 02:43:44.411229 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:44.411231 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:44.411281 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:44.411360 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:44.411824 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:44.412381 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:44.413185 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:44.413402 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:44.413500 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:44.416339 (+ 2839us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.157.130:35455: BlockingRecv error: recv got EOF from 127.13.157.130:35455 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Sep 25 02:43:44 dist-test-slave-kznw krb5kdc[22255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.157.131@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 02:43:44 dist-test-slave-kznw krb5kdc[22255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.157.131@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 02:43:44.599922 23276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:44.594518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:45019 (local address 127.13.157.131:38283)
0925 02:43:44.594783 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:44.594796 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:44.594896 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:44.595195 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:44.595198 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:44.595365 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:44.595460 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:44.595465 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:44.596235 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:44.596237 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:44.596788 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:44.596795 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:44.597282 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:44.597285 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:44.597346 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:44.597815 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:44.597829 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:44.599752 (+ 1923us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.157.190:45019: Server oryx/127.13.157.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":90,"threads_started":1}
W20260925 02:43:44.600066 23274 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:43:44.594565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.131:38283 (local address 127.13.157.190:45019)
0925 02:43:44.594700 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:44.594703 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:44.594887 (+ 184us) server_negotiation.cc:407] Connection header received
0925 02:43:44.595044 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:44.595046 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:44.595097 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:44.595170 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:44.595572 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:44.596146 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:44.596899 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:44.597164 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:44.597243 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:44.599910 (+ 2667us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.157.131:38283: BlockingRecv error: recv got EOF from 127.13.157.131:38283 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Sep 25 02:43:45 dist-test-slave-kznw krb5kdc[22255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.157.129@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 02:43:45 dist-test-slave-kznw krb5kdc[22255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.157.129@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 02:43:45.235926 23277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:45.230379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:45019 (local address 127.13.157.129:51219)
0925 02:43:45.230680 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:45.230693 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:45.230797 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:45.231150 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:45.231152 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:45.231334 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:45.231464 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:45.231473 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:45.232286 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:45.232289 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:45.232907 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:45.232917 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:45.233421 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:45.233424 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:45.233522 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:45.234009 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:45.234023 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:45.235807 (+ 1784us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.157.190:45019: Server oryx/127.13.157.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":92,"threads_started":1}
W20260925 02:43:45.236075 23278 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:43:45.230456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.129:51219 (local address 127.13.157.190:45019)
0925 02:43:45.230713 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:45.230717 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:45.230766 (+ 49us) server_negotiation.cc:407] Connection header received
0925 02:43:45.230963 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:45.230967 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:45.231027 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:45.231110 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:45.231619 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:45.232176 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:45.233042 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:45.233302 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:45.233421 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:45.235943 (+ 2522us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.157.129:51219: BlockingRecv error: recv got EOF from 127.13.157.129:51219 (error 108)
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":57,"threads_started":1}
Sep 25 02:43:45 dist-test-slave-kznw krb5kdc[22255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.157.130@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 02:43:45 dist-test-slave-kznw krb5kdc[22255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.157.130@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 02:43:45.422376 23279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:45.417124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:45019 (local address 127.13.157.130:35987)
0925 02:43:45.417392 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:45.417405 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:45.417511 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:45.417717 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:45.417719 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:45.417888 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:45.417991 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:45.417998 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:45.418755 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:45.418758 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:45.419374 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:45.419385 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:45.419827 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:45.419829 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:45.419891 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:45.420372 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:45.420383 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:45.422222 (+ 1839us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.157.190:45019: Server oryx/127.13.157.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":102,"threads_started":1}
W20260925 02:43:45.422468 23278 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:43:45.417184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.130:35987 (local address 127.13.157.190:45019)
0925 02:43:45.417353 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:45.417358 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:45.417471 (+ 113us) server_negotiation.cc:407] Connection header received
0925 02:43:45.417581 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:45.417584 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:45.417629 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:45.417695 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:45.418105 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:45.418659 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:45.419496 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:45.419733 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:45.419811 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:45.422367 (+ 2556us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.157.130:35987: BlockingRecv error: recv got EOF from 127.13.157.130:35987 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Sep 25 02:43:45 dist-test-slave-kznw krb5kdc[22255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.157.131@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 02:43:45 dist-test-slave-kznw krb5kdc[22255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.157.131@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 02:43:45.606474 23280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:45.600732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:45019 (local address 127.13.157.131:40803)
0925 02:43:45.601043 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:45.601059 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:45.601191 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:45.601524 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:45.601527 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:45.601748 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:45.601858 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:45.601865 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:45.602748 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:45.602750 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:45.603347 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:45.603357 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:45.603817 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:45.603821 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:45.603899 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:45.604454 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:45.604468 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:45.606317 (+ 1849us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.157.190:45019: Server oryx/127.13.157.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":100,"threads_started":1}
W20260925 02:43:45.606580 23278 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:43:45.600818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.131:40803 (local address 127.13.157.190:45019)
0925 02:43:45.600955 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:45.600960 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:45.601168 (+ 208us) server_negotiation.cc:407] Connection header received
0925 02:43:45.601287 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:45.601290 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:45.601349 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:45.601432 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:45.601963 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:45.602648 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:45.603444 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:45.603673 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:45.603749 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:45.606454 (+ 2705us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.157.131:40803: BlockingRecv error: recv got EOF from 127.13.157.131:40803 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Sep 25 02:43:46 dist-test-slave-kznw krb5kdc[22255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.157.129@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 02:43:46 dist-test-slave-kznw krb5kdc[22255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.157.129@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 02:43:46.254792 23281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:46.236752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:45019 (local address 127.13.157.129:39811)
0925 02:43:46.237038 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:46.237052 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:46.237141 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:46.237363 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:46.237365 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:46.237564 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:46.237661 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:46.237668 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:46.238478 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:46.238482 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:46.239201 (+ 719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:46.239218 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:46.239714 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:46.239718 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:46.239789 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:46.252732 (+ 12943us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:46.252748 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:46.254661 (+ 1913us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.157.190:45019: Server oryx/127.13.157.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":86,"threads_started":1}
W20260925 02:43:46.255055 23282 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:43:46.236829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.129:39811 (local address 127.13.157.190:45019)
0925 02:43:46.237117 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:46.237121 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:46.237134 (+ 13us) server_negotiation.cc:407] Connection header received
0925 02:43:46.237215 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:46.237217 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:46.237261 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:46.237338 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:46.237800 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:46.238345 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:46.239347 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:46.239592 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:46.239694 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:46.254869 (+ 15175us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.157.129:39811: BlockingRecv error: recv got EOF from 127.13.157.129:39811 (error 108)
Metrics: {"server-negotiator.queue_time_us":221,"thread_start_us":58,"threads_started":1}
Sep 25 02:43:46 dist-test-slave-kznw krb5kdc[22255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.157.130@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 02:43:46 dist-test-slave-kznw krb5kdc[22255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.157.130@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 02:43:46.428679 23283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:46.423295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:45019 (local address 127.13.157.130:52735)
0925 02:43:46.423582 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:46.423594 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:46.423685 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:46.423878 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:46.423880 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:46.424057 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:46.424152 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:46.424157 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:46.424843 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:46.424845 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:46.425412 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:46.425421 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:46.425896 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:46.425899 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:46.425977 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:46.426432 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:46.426444 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:46.428554 (+ 2110us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.157.190:45019: Server oryx/127.13.157.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":83,"threads_started":1}
W20260925 02:43:46.428851 23282 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:43:46.423379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.130:52735 (local address 127.13.157.190:45019)
0925 02:43:46.423519 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:46.423522 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:46.423643 (+ 121us) server_negotiation.cc:407] Connection header received
0925 02:43:46.423747 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:46.423749 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:46.423790 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:46.423856 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:46.424237 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:46.424756 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:46.425541 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:46.425769 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:46.425854 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:46.428732 (+ 2878us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.157.130:52735: BlockingRecv error: recv got EOF from 127.13.157.130:52735 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Sep 25 02:43:46 dist-test-slave-kznw krb5kdc[22255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.157.131@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 02:43:46 dist-test-slave-kznw krb5kdc[22255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.157.131@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 02:43:46.612709 23284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:46.607366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:45019 (local address 127.13.157.131:34369)
0925 02:43:46.607700 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:46.607716 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:46.607829 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:46.608133 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:46.608136 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:46.608336 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:46.608433 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:46.608439 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:46.609226 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:46.609230 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:46.609800 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:46.609810 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:46.610259 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:46.610263 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:46.610345 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:46.610827 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:46.610838 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:46.612547 (+ 1709us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.157.190:45019: Server oryx/127.13.157.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":114,"threads_started":1}
W20260925 02:43:46.612788 23282 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:43:46.607489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.131:34369 (local address 127.13.157.190:45019)
0925 02:43:46.607675 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:46.607680 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:46.607791 (+ 111us) server_negotiation.cc:407] Connection header received
0925 02:43:46.607919 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:46.607922 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:46.608017 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:46.608088 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:46.608591 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:46.609119 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:46.609927 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:46.610153 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:46.610239 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:46.612685 (+ 2446us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.157.131:34369: BlockingRecv error: recv got EOF from 127.13.157.131:34369 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Sep 25 02:43:47 dist-test-slave-kznw krb5kdc[22255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.157.129@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 02:43:47 dist-test-slave-kznw krb5kdc[22255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.157.129@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 02:43:47.261566 23285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:47.255733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:45019 (local address 127.13.157.129:35889)
0925 02:43:47.256034 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:47.256052 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:47.256179 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:47.256479 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:47.256482 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:47.256700 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:47.256839 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:47.256850 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:47.257805 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:47.257808 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:47.258511 (+ 703us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:47.258523 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:47.259001 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:47.259003 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:47.259066 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:47.259546 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:47.259558 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:47.261409 (+ 1851us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.157.190:45019: Server oryx/127.13.157.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":92,"threads_started":1}
W20260925 02:43:47.261679 23286 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:43:47.255785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.129:35889 (local address 127.13.157.190:45019)
0925 02:43:47.256036 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:47.256040 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:47.256112 (+ 72us) server_negotiation.cc:407] Connection header received
0925 02:43:47.256275 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:47.256278 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:47.256335 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:47.256428 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:47.257022 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:47.257696 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:47.258626 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:47.258860 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:47.258984 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:47.261562 (+ 2578us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.157.129:35889: BlockingRecv error: recv got EOF from 127.13.157.129:35889 (error 108)
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":71,"threads_started":1}
Sep 25 02:43:47 dist-test-slave-kznw krb5kdc[22255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.157.130@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 02:43:47 dist-test-slave-kznw krb5kdc[22255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.157.130@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 02:43:47.435060 23287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:47.429547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:45019 (local address 127.13.157.130:37065)
0925 02:43:47.429844 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:47.429858 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:47.430002 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:47.430292 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:47.430294 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:47.430523 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:47.430624 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:47.430630 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:47.431367 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:47.431369 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:47.432015 (+ 646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:47.432025 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:47.432409 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:47.432411 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:47.432483 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:47.432925 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:47.432937 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:47.434900 (+ 1963us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.157.190:45019: Server oryx/127.13.157.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":109,"threads_started":1}
W20260925 02:43:47.435199 23286 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:43:47.429680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.130:37065 (local address 127.13.157.190:45019)
0925 02:43:47.429848 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:47.429853 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:47.429931 (+ 78us) server_negotiation.cc:407] Connection header received
0925 02:43:47.430105 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:47.430107 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:47.430166 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:47.430277 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:47.430718 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:47.431267 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:47.432114 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:47.432327 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:47.432400 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:47.435082 (+ 2682us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.157.130:37065: BlockingRecv error: recv got EOF from 127.13.157.130:37065 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Sep 25 02:43:47 dist-test-slave-kznw krb5kdc[22255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.157.131@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 02:43:47 dist-test-slave-kznw krb5kdc[22255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.157.131@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 02:43:47.618832 23288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:47.613520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:45019 (local address 127.13.157.131:42521)
0925 02:43:47.613784 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:47.613797 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:47.613905 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:47.614156 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:47.614159 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:47.614356 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:47.614451 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:47.614457 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:47.615332 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:47.615336 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:47.615892 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:47.615900 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:47.616339 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:47.616343 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:47.616424 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:47.616889 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:47.616902 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:47.618680 (+ 1778us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.157.190:45019: Server oryx/127.13.157.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":84,"threads_started":1}
W20260925 02:43:47.618963 23286 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:43:47.613570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.131:42521 (local address 127.13.157.190:45019)
0925 02:43:47.613738 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:47.613742 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:47.613865 (+ 123us) server_negotiation.cc:407] Connection header received
0925 02:43:47.613990 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:47.613993 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:47.614045 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:47.614119 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:47.614592 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:47.615230 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:47.616004 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:47.616235 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:47.616319 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:47.618820 (+ 2501us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.157.131:42521: BlockingRecv error: recv got EOF from 127.13.157.131:42521 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Sep 25 02:43:48 dist-test-slave-kznw krb5kdc[22255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.157.129@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 02:43:48 dist-test-slave-kznw krb5kdc[22255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.157.129@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 02:43:48.267727 23289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:48.262387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:45019 (local address 127.13.157.129:38605)
0925 02:43:48.262730 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:48.262743 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:48.262836 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:48.263092 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:48.263094 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:48.263267 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:48.263363 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:48.263368 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:48.264165 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:48.264169 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:48.264746 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:48.264756 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:48.265238 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:48.265241 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:48.265308 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:48.265795 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:48.265806 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:48.267607 (+ 1801us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.157.190:45019: Server oryx/127.13.157.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":94,"threads_started":1}
W20260925 02:43:48.267866 23290 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:43:48.262529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.129:38605 (local address 127.13.157.190:45019)
0925 02:43:48.262786 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:48.262790 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:48.262807 (+ 17us) server_negotiation.cc:407] Connection header received
0925 02:43:48.262933 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:48.262935 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:48.262980 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:48.263060 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:48.263471 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:48.264055 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:48.264853 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:48.265125 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:48.265214 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:48.267740 (+ 2526us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.157.129:38605: BlockingRecv error: recv got EOF from 127.13.157.129:38605 (error 108)
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":74,"threads_started":1}
Sep 25 02:43:48 dist-test-slave-kznw krb5kdc[22255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.157.130@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 02:43:48 dist-test-slave-kznw krb5kdc[22255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.157.130@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 02:43:48.441445 23291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:48.435970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:45019 (local address 127.13.157.130:34455)
0925 02:43:48.436283 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:48.436296 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:48.436393 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:48.436581 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:48.436583 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:48.436757 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:48.436850 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:48.436855 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:48.437641 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:48.437644 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:48.438253 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:48.438263 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:48.438712 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:48.438716 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:48.438787 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:48.439323 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:48.439335 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:48.441320 (+ 1985us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.157.190:45019: Server oryx/127.13.157.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":78,"threads_started":1}
W20260925 02:43:48.441634 23290 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:43:48.436074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.130:34455 (local address 127.13.157.190:45019)
0925 02:43:48.436208 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:48.436211 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:48.436354 (+ 143us) server_negotiation.cc:407] Connection header received
0925 02:43:48.436459 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:48.436461 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:48.436505 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:48.436570 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:48.436950 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:48.437524 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:48.438365 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:48.438590 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:48.438668 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:48.441510 (+ 2842us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.157.130:34455: BlockingRecv error: recv got EOF from 127.13.157.130:34455 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Sep 25 02:43:48 dist-test-slave-kznw krb5kdc[22255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.157.131@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 02:43:48 dist-test-slave-kznw krb5kdc[22255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.157.131@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 02:43:48.625262 23292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:48.619712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:45019 (local address 127.13.157.131:45177)
0925 02:43:48.620083 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:48.620099 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:48.620218 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:48.620464 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:48.620467 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:48.620669 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:48.620766 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:48.620771 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:48.621544 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:48.621546 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:48.622122 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:48.622130 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:48.622586 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:48.622589 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:48.622670 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:48.623249 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:48.623264 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:48.625133 (+ 1869us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.157.190:45019: Server oryx/127.13.157.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":90,"threads_started":1}
W20260925 02:43:48.625401 23290 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:43:48.619809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.131:45177 (local address 127.13.157.190:45019)
0925 02:43:48.619997 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:48.620002 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:48.620172 (+ 170us) server_negotiation.cc:407] Connection header received
0925 02:43:48.620308 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:48.620311 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:48.620359 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:48.620426 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:48.620883 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:48.621412 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:48.622254 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:48.622478 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:48.622569 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:48.625286 (+ 2717us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.157.131:45177: BlockingRecv error: recv got EOF from 127.13.157.131:45177 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Sep 25 02:43:49 dist-test-slave-kznw krb5kdc[22255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.157.129@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 02:43:49 dist-test-slave-kznw krb5kdc[22255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.157.129@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 02:43:49.273782 23293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:49.268525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:45019 (local address 127.13.157.129:34967)
0925 02:43:49.268823 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:49.268837 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:49.268933 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:49.269200 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:49.269202 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:49.269375 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:49.269470 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:49.269476 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:49.270250 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:49.270253 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:49.270813 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:49.270822 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:49.271355 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:49.271358 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:49.271422 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:49.271908 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:49.271920 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:49.273662 (+ 1742us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.157.190:45019: Server oryx/127.13.157.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":82,"threads_started":1}
W20260925 02:43:49.273944 23294 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:43:49.268703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.129:34967 (local address 127.13.157.190:45019)
0925 02:43:49.268924 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:49.268927 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:49.268941 (+ 14us) server_negotiation.cc:407] Connection header received
0925 02:43:49.269050 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:49.269052 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:49.269095 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:49.269169 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:49.269576 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:49.270145 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:49.271004 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:49.271244 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:49.271329 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:49.273804 (+ 2475us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.157.129:34967: BlockingRecv error: recv got EOF from 127.13.157.129:34967 (error 108)
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":66,"threads_started":1}
Sep 25 02:43:49 dist-test-slave-kznw krb5kdc[22255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.157.130@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 02:43:49 dist-test-slave-kznw krb5kdc[22255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.157.130@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 02:43:49.447535 23295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:49.442188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:45019 (local address 127.13.157.130:59103)
0925 02:43:49.442503 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:49.442517 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:49.442611 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:49.442833 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:49.442835 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:49.443042 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:49.443144 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:49.443150 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:49.443982 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:49.443985 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:49.444556 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:49.444564 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:49.445031 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:49.445035 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:49.445101 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:49.445560 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:49.445571 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:49.447407 (+ 1836us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.157.190:45019: Server oryx/127.13.157.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":99,"threads_started":1}
W20260925 02:43:49.447695 23294 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:43:49.442290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.130:59103 (local address 127.13.157.190:45019)
0925 02:43:49.442439 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:49.442443 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:49.442574 (+ 131us) server_negotiation.cc:407] Connection header received
0925 02:43:49.442700 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:49.442702 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:49.442750 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:49.442808 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:49.443231 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:49.443892 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:49.444677 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:49.444915 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:49.445000 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:49.447551 (+ 2551us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.157.130:59103: BlockingRecv error: recv got EOF from 127.13.157.130:59103 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Sep 25 02:43:49 dist-test-slave-kznw krb5kdc[22255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.157.131@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 02:43:49 dist-test-slave-kznw krb5kdc[22255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.157.131@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 02:43:49.631659 23296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:49.626005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:45019 (local address 127.13.157.131:47589)
0925 02:43:49.626387 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:49.626402 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:49.626525 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:49.626768 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:49.626771 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:49.627014 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:49.627113 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:49.627120 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:49.627867 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:49.627871 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:49.628434 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:49.628444 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:49.628894 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:49.628898 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:49.628977 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:49.629494 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:49.629506 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:49.631498 (+ 1992us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.157.190:45019: Server oryx/127.13.157.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":95,"threads_started":1}
W20260925 02:43:49.631934 23294 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:43:49.626086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.131:47589 (local address 127.13.157.190:45019)
0925 02:43:49.626239 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:49.626244 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:49.626487 (+ 243us) server_negotiation.cc:407] Connection header received
0925 02:43:49.626613 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:49.626616 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:49.626666 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:49.626730 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:49.627230 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:49.627766 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:49.628553 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:49.628785 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:49.628874 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:49.631643 (+ 2769us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.157.131:47589: BlockingRecv error: recv got EOF from 127.13.157.131:47589 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Sep 25 02:43:50 dist-test-slave-kznw krb5kdc[22255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.157.129@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 02:43:50 dist-test-slave-kznw krb5kdc[22255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.157.129@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 02:43:50.279953 23297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:50.274523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:45019 (local address 127.13.157.129:35763)
0925 02:43:50.274832 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:50.274846 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:50.274970 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:50.275255 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:50.275257 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:50.275429 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:50.275531 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:50.275537 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:50.276391 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:50.276394 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:50.276953 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:50.276962 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:50.277396 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:50.277399 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:50.277454 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:50.277903 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:50.277913 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:50.279802 (+ 1889us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.157.190:45019: Server oryx/127.13.157.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":99,"threads_started":1}
W20260925 02:43:50.280064 23298 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:43:50.274644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.129:35763 (local address 127.13.157.190:45019)
0925 02:43:50.274886 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:50.274889 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:50.274928 (+ 39us) server_negotiation.cc:407] Connection header received
0925 02:43:50.275103 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:50.275105 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:50.275151 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:50.275224 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:50.275672 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:50.276277 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:50.277082 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:50.277311 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:50.277395 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:50.279950 (+ 2555us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.157.129:35763: BlockingRecv error: recv got EOF from 127.13.157.129:35763 (error 108)
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":83,"threads_started":1}
Sep 25 02:43:50 dist-test-slave-kznw krb5kdc[22255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.157.130@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 02:43:50 dist-test-slave-kznw krb5kdc[22255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.157.130@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 02:43:50.453989 23299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:50.448382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:45019 (local address 127.13.157.130:49947)
0925 02:43:50.448688 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:50.448701 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:50.448820 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:50.449146 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:50.449148 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:50.449368 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:50.449474 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:50.449481 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:50.450270 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:50.450273 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:50.450854 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:50.450862 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:50.451389 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:50.451393 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:50.451476 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:50.452002 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:50.452013 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:50.453846 (+ 1833us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.157.190:45019: Server oryx/127.13.157.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":94,"threads_started":1}
W20260925 02:43:50.454119 23298 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:43:50.448455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.130:49947 (local address 127.13.157.190:45019)
0925 02:43:50.448631 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:50.448636 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:50.448772 (+ 136us) server_negotiation.cc:407] Connection header received
0925 02:43:50.448911 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:50.448914 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:50.449017 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:50.449102 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:50.449592 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:50.450176 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:50.451011 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:50.451254 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:50.451334 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:50.454015 (+ 2681us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.157.130:49947: BlockingRecv error: recv got EOF from 127.13.157.130:49947 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Sep 25 02:43:50 dist-test-slave-kznw krb5kdc[22255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.157.131@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 02:43:50 dist-test-slave-kznw krb5kdc[22255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.157.131@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 02:43:50.638356 23300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:50.632396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:45019 (local address 127.13.157.131:56909)
0925 02:43:50.632686 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:50.632699 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:50.632804 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:50.633139 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:50.633143 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:50.633332 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:50.633451 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:50.633457 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:50.634406 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:50.634409 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:50.635150 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:50.635162 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:50.635716 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:50.635719 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:50.635785 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:50.636329 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:50.636341 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:50.638206 (+ 1865us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.157.190:45019: Server oryx/127.13.157.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":107,"threads_started":1}
W20260925 02:43:50.638466 23298 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:43:50.632472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.131:56909 (local address 127.13.157.190:45019)
0925 02:43:50.632624 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:50.632628 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:50.632762 (+ 134us) server_negotiation.cc:407] Connection header received
0925 02:43:50.632901 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:50.632904 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:50.632954 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:50.633015 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:50.633599 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:50.634266 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:50.635273 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:50.635587 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:50.635687 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:50.638365 (+ 2678us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.157.131:56909: BlockingRecv error: recv got EOF from 127.13.157.131:56909 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Sep 25 02:43:51 dist-test-slave-kznw krb5kdc[22255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.157.129@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 02:43:51 dist-test-slave-kznw krb5kdc[22255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.157.129@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 02:43:51.289162 23301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:51.280698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:45019 (local address 127.13.157.129:57049)
0925 02:43:51.281030 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:51.281044 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:51.281149 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:51.281373 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:51.281375 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:51.281553 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:51.281649 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:51.281654 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:51.282476 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:51.282479 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:51.283103 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:51.283114 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:51.283563 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:51.283565 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:51.283630 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:51.287107 (+ 3477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:51.287120 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:51.289029 (+ 1909us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.157.190:45019: Server oryx/127.13.157.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":88,"threads_started":1}
W20260925 02:43:51.289371 23302 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:43:51.280778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.129:57049 (local address 127.13.157.190:45019)
0925 02:43:51.281011 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:51.281014 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:51.281150 (+ 136us) server_negotiation.cc:407] Connection header received
0925 02:43:51.281216 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:51.281218 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:51.281272 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:51.281352 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:51.281758 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:51.282367 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:51.283220 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:51.283454 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:51.283540 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:51.289222 (+ 5682us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.157.129:57049: BlockingRecv error: recv got EOF from 127.13.157.129:57049 (error 108)
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":58,"threads_started":1}
Sep 25 02:43:51 dist-test-slave-kznw krb5kdc[22255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.157.130@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 02:43:51 dist-test-slave-kznw krb5kdc[22255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.157.130@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 02:43:51.460292 23303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:51.454774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:45019 (local address 127.13.157.130:34293)
0925 02:43:51.455074 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:51.455091 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:51.455205 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:51.455467 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:51.455470 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:51.455681 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:51.455797 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:51.455805 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:51.456646 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:51.456650 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:51.457284 (+ 634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:51.457294 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:51.457780 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:51.457784 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:51.457864 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:51.458301 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:51.458312 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:51.460165 (+ 1853us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.157.190:45019: Server oryx/127.13.157.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":73,"threads_started":1}
W20260925 02:43:51.460466 23302 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:43:51.454838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.130:34293 (local address 127.13.157.190:45019)
0925 02:43:51.455006 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:51.455011 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:51.455176 (+ 165us) server_negotiation.cc:407] Connection header received
0925 02:43:51.455294 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:51.455297 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:51.455347 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:51.455422 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:51.455916 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:51.456538 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:51.457395 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:51.457673 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:51.457765 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:51.460332 (+ 2567us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.157.130:34293: BlockingRecv error: recv got EOF from 127.13.157.130:34293 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Sep 25 02:43:51 dist-test-slave-kznw krb5kdc[22255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.157.131@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 02:43:51 dist-test-slave-kznw krb5kdc[22255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.157.131@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 02:43:51.644450 23304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:51.639191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:45019 (local address 127.13.157.131:52393)
0925 02:43:51.639501 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:51.639517 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:51.639647 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:51.639899 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:51.639902 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:51.640143 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:51.640245 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:51.640251 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:51.641030 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:51.641034 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:51.641587 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:51.641595 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:51.642031 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:51.642035 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:51.642118 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:51.642558 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:51.642570 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:51.644301 (+ 1731us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.157.190:45019: Server oryx/127.13.157.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":87,"threads_started":1}
W20260925 02:43:51.644557 23302 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:43:51.639333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.131:52393 (local address 127.13.157.190:45019)
0925 02:43:51.639502 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:51.639507 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:51.639588 (+ 81us) server_negotiation.cc:407] Connection header received
0925 02:43:51.639738 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:51.639741 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:51.639792 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:51.639862 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:51.640369 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:51.640922 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:51.641700 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:51.641925 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:51.642013 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:51.644457 (+ 2444us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.157.131:52393: BlockingRecv error: recv got EOF from 127.13.157.131:52393 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Sep 25 02:43:52 dist-test-slave-kznw krb5kdc[22255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.157.129@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 02:43:52 dist-test-slave-kznw krb5kdc[22255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.157.129@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 02:43:52.295504 23305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:52.290033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:45019 (local address 127.13.157.129:53557)
0925 02:43:52.290323 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:52.290337 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:52.290431 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:52.290655 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:52.290657 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:52.290832 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:52.290959 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:52.290965 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:52.291777 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:52.291780 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:52.292354 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:52.292363 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:52.292829 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:52.292832 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:52.292896 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:52.293398 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:52.293409 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:52.295374 (+ 1965us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.157.190:45019: Server oryx/127.13.157.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":91,"threads_started":1}
W20260925 02:43:52.295696 23306 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:43:52.290106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.129:53557 (local address 127.13.157.190:45019)
0925 02:43:52.290381 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:52.290384 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:52.290397 (+ 13us) server_negotiation.cc:407] Connection header received
0925 02:43:52.290510 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:52.290512 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:52.290553 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:52.290627 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:52.291094 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:52.291671 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:52.292483 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:52.292721 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:52.292799 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:52.295543 (+ 2744us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.157.129:53557: BlockingRecv error: recv got EOF from 127.13.157.129:53557 (error 108)
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":81,"threads_started":1}
Sep 25 02:43:52 dist-test-slave-kznw krb5kdc[22255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.157.130@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 02:43:52 dist-test-slave-kznw krb5kdc[22255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.157.130@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 02:43:52.466205 23307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:52.460965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:45019 (local address 127.13.157.130:47055)
0925 02:43:52.461229 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:52.461244 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:52.461347 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:52.461572 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:52.461574 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:52.461746 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:52.461848 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:52.461853 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:52.462604 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:52.462607 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:52.463216 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:52.463225 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:52.463613 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:52.463615 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:52.463678 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:52.464152 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:52.464164 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:52.466054 (+ 1890us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.157.190:45019: Server oryx/127.13.157.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":90,"threads_started":1}
W20260925 02:43:52.466282 23306 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:43:52.461039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.130:47055 (local address 127.13.157.190:45019)
0925 02:43:52.461209 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:52.461212 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:52.461298 (+ 86us) server_negotiation.cc:407] Connection header received
0925 02:43:52.461418 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:52.461420 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:52.461476 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:52.461538 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:52.461938 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:52.462485 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:52.463305 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:52.463531 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:52.463602 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:52.466184 (+ 2582us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.157.130:47055: BlockingRecv error: recv got EOF from 127.13.157.130:47055 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Sep 25 02:43:52 dist-test-slave-kznw krb5kdc[22255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.157.131@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 02:43:52 dist-test-slave-kznw krb5kdc[22255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.157.131@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 02:43:52.650671 23308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:52.645246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:45019 (local address 127.13.157.131:37437)
0925 02:43:52.645494 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:52.645507 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:52.645600 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:52.645801 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:52.645803 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:52.646013 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:52.646114 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:52.646119 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:52.647018 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:52.647022 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:52.647607 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:52.647619 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:52.648095 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:52.648099 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:52.648183 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:52.648704 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:52.648717 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:52.650549 (+ 1832us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.157.190:45019: Server oryx/127.13.157.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":73,"threads_started":1}
W20260925 02:43:52.650827 23306 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:43:52.645325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.131:37437 (local address 127.13.157.190:45019)
0925 02:43:52.645466 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:52.645469 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:52.645553 (+ 84us) server_negotiation.cc:407] Connection header received
0925 02:43:52.645665 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:52.645667 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:52.645711 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:52.645779 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:52.646229 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:52.646836 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:52.647759 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:52.647983 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:52.648070 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:52.650709 (+ 2639us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.157.131:37437: BlockingRecv error: recv got EOF from 127.13.157.131:37437 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Sep 25 02:43:53 dist-test-slave-kznw krb5kdc[22255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.157.129@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 02:43:53 dist-test-slave-kznw krb5kdc[22255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.157.129@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 02:43:53.301952 23309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:53.296405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:45019 (local address 127.13.157.129:40697)
0925 02:43:53.296710 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:53.296723 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:53.296822 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:53.297088 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:53.297090 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:53.297268 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:53.297364 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:53.297369 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:53.298156 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:53.298159 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:53.298731 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:53.298740 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:53.299302 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:53.299307 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:53.299393 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:53.299941 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:53.299956 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:53.301804 (+ 1848us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.157.190:45019: Server oryx/127.13.157.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":92,"threads_started":1}
W20260925 02:43:53.302076 23310 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:43:53.296554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.129:40697 (local address 127.13.157.190:45019)
0925 02:43:53.296770 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:53.296773 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:53.296787 (+ 14us) server_negotiation.cc:407] Connection header received
0925 02:43:53.296904 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:53.296906 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:53.296979 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:53.297056 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:53.297465 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:53.298042 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:53.298856 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:53.299177 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:53.299279 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:53.301946 (+ 2667us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.157.129:40697: BlockingRecv error: recv got EOF from 127.13.157.129:40697 (error 108)
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":59,"threads_started":1}
Sep 25 02:43:53 dist-test-slave-kznw krb5kdc[22255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.157.130@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 02:43:53 dist-test-slave-kznw krb5kdc[22255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.157.130@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 02:43:53.472182 23311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:53.467017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:45019 (local address 127.13.157.130:45725)
0925 02:43:53.467300 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:53.467313 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:53.467454 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:53.467688 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:53.467691 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:53.467892 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:53.467995 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:53.468001 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:53.468779 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:53.468782 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:53.469347 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:53.469354 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:53.469770 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:53.469773 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:53.469866 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:53.470348 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:53.470360 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:53.472031 (+ 1671us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.157.190:45019: Server oryx/127.13.157.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":81,"threads_started":1}
W20260925 02:43:53.472278 23310 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:43:53.467105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.130:45725 (local address 127.13.157.190:45019)
0925 02:43:53.467258 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:53.467263 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:53.467387 (+ 124us) server_negotiation.cc:407] Connection header received
0925 02:43:53.467541 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:53.467545 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:53.467592 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:53.467656 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:53.468109 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:53.468678 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:53.469458 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:53.469674 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:53.469757 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:53.472169 (+ 2412us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.157.130:45725: BlockingRecv error: recv got EOF from 127.13.157.130:45725 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Sep 25 02:43:53 dist-test-slave-kznw krb5kdc[22255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.157.131@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 02:43:53 dist-test-slave-kznw krb5kdc[22255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.157.131@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 02:43:53.657125 23312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:53.651490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:45019 (local address 127.13.157.131:52361)
0925 02:43:53.651796 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:53.651821 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:53.651948 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:53.652253 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:53.652256 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:53.652455 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:53.652574 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:53.652581 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:53.653335 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:53.653338 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:53.653907 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:53.653915 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:53.654354 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:53.654358 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:53.654439 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:53.654981 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:53.654992 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:53.656935 (+ 1943us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.157.190:45019: Server oryx/127.13.157.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":88,"threads_started":1}
W20260925 02:43:53.657166 23310 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:43:53.651607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.131:52361 (local address 127.13.157.190:45019)
0925 02:43:53.651795 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:53.651800 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:53.651897 (+ 97us) server_negotiation.cc:407] Connection header received
0925 02:43:53.652091 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:53.652093 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:53.652143 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:53.652209 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:53.652695 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:53.653231 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:53.654024 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:53.654242 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:53.654339 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:53.657029 (+ 2690us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.157.131:52361: BlockingRecv error: recv got EOF from 127.13.157.131:52361 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
Sep 25 02:43:54 dist-test-slave-kznw krb5kdc[22255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.157.129@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 02:43:54 dist-test-slave-kznw krb5kdc[22255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.157.129@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 02:43:54.308269 23313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:54.302748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:45019 (local address 127.13.157.129:40269)
0925 02:43:54.303112 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:54.303125 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:54.303231 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:54.303514 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:54.303517 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:54.303696 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:54.303790 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:54.303795 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:54.304624 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:54.304627 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:54.305201 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:54.305210 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:54.305674 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:54.305677 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:54.305741 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:54.306259 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:54.306271 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:54.308112 (+ 1841us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.157.190:45019: Server oryx/127.13.157.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":89,"threads_started":1}
W20260925 02:43:54.308377 23314 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:43:54.302828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.129:40269 (local address 127.13.157.190:45019)
0925 02:43:54.303105 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:54.303108 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:54.303187 (+ 79us) server_negotiation.cc:407] Connection header received
0925 02:43:54.303301 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:54.303304 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:54.303354 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:54.303437 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:54.303967 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:54.304512 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:54.305315 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:54.305558 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:54.305639 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:54.308256 (+ 2617us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.157.129:40269: BlockingRecv error: recv got EOF from 127.13.157.129:40269 (error 108)
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":96,"threads_started":1}
Sep 25 02:43:54 dist-test-slave-kznw krb5kdc[22255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.157.130@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 02:43:54 dist-test-slave-kznw krb5kdc[22255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.157.130@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 02:43:54.478312 23315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:54.473048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:45019 (local address 127.13.157.130:41349)
0925 02:43:54.473311 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:54.473325 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:54.473431 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:54.473655 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:54.473657 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:54.473834 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:54.473933 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:54.473938 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:54.474720 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:54.474723 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:54.475354 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:54.475363 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:54.475852 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:54.475855 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:54.475922 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:54.476417 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:54.476428 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:54.478190 (+ 1762us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.157.190:45019: Server oryx/127.13.157.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":70,"threads_started":1}
W20260925 02:43:54.478506 23314 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:43:54.473162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.130:41349 (local address 127.13.157.190:45019)
0925 02:43:54.473327 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:54.473331 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:54.473393 (+ 62us) server_negotiation.cc:407] Connection header received
0925 02:43:54.473507 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:54.473510 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:54.473556 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:54.473627 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:54.474060 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:54.474600 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:54.475490 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:54.475741 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:54.475840 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:54.478356 (+ 2516us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.157.130:41349: BlockingRecv error: recv got EOF from 127.13.157.130:41349 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Sep 25 02:43:54 dist-test-slave-kznw krb5kdc[22255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.157.131@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 02:43:54 dist-test-slave-kznw krb5kdc[22255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.157.131@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 02:43:54.663460 23316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:54.658036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:45019 (local address 127.13.157.131:46369)
0925 02:43:54.658294 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:54.658306 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:54.658416 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:54.658693 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:54.658696 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:54.658896 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:54.659037 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:54.659052 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:54.659812 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:54.659816 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:54.660386 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:54.660395 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:54.660835 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:54.660839 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:54.660924 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:54.661426 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:54.661438 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:54.663340 (+ 1902us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.157.190:45019: Server oryx/127.13.157.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":85,"threads_started":1}
W20260925 02:43:54.663616 23314 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:43:54.658086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.131:46369 (local address 127.13.157.190:45019)
0925 02:43:54.658223 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:54.658228 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:54.658371 (+ 143us) server_negotiation.cc:407] Connection header received
0925 02:43:54.658505 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:54.658508 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:54.658586 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:54.658651 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:54.659152 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:54.659703 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:54.660494 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:54.660726 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:54.660815 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:54.663499 (+ 2684us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.157.131:46369: BlockingRecv error: recv got EOF from 127.13.157.131:46369 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Sep 25 02:43:55 dist-test-slave-kznw krb5kdc[22255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.157.129@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 02:43:55 dist-test-slave-kznw krb5kdc[22255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.157.129@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 02:43:55.315097 23317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:55.309223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:45019 (local address 127.13.157.129:46125)
0925 02:43:55.309586 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:55.309602 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:55.309708 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:55.309957 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:55.309959 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:55.310148 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:55.310249 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:55.310255 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:55.311330 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:55.311332 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:55.312061 (+ 729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:55.312073 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:55.312515 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:55.312519 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:55.312593 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:55.313082 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:55.313094 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:55.314941 (+ 1847us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.157.190:45019: Server oryx/127.13.157.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":118,"threads_started":1}
W20260925 02:43:55.315196 23318 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:43:55.309392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.129:46125 (local address 127.13.157.190:45019)
0925 02:43:55.309718 (+ 326us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:55.309722 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:55.309759 (+ 37us) server_negotiation.cc:407] Connection header received
0925 02:43:55.309795 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:55.309797 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:55.309840 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:55.309907 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:55.310400 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:55.311195 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:55.312172 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:55.312398 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:55.312484 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:55.315079 (+ 2595us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.157.129:46125: BlockingRecv error: recv got EOF from 127.13.157.129:46125 (error 108)
Metrics: {"server-negotiator.queue_time_us":257,"thread_start_us":87,"threads_started":1}
Sep 25 02:43:55 dist-test-slave-kznw krb5kdc[22255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.157.130@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 02:43:55 dist-test-slave-kznw krb5kdc[22255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.157.130@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 02:43:55.484509 23319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:55.479168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:45019 (local address 127.13.157.130:58281)
0925 02:43:55.479425 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:55.479440 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:55.479549 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:55.479783 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:55.479785 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:55.479976 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:55.480074 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:55.480079 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:55.480827 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:55.480830 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:55.481464 (+ 634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:55.481474 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:55.481906 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:55.481909 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:55.482016 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:55.482543 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:55.482555 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:55.484328 (+ 1773us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.157.190:45019: Server oryx/127.13.157.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":74,"threads_started":1}
W20260925 02:43:55.484581 23318 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:43:55.479255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.130:58281 (local address 127.13.157.190:45019)
0925 02:43:55.479434 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:55.479437 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:55.479503 (+ 66us) server_negotiation.cc:407] Connection header received
0925 02:43:55.479635 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:55.479637 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:55.479682 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:55.479750 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:55.480200 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:55.480726 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:55.481585 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:55.481807 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:55.481888 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:55.484477 (+ 2589us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.157.130:58281: BlockingRecv error: recv got EOF from 127.13.157.130:58281 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Sep 25 02:43:55 dist-test-slave-kznw krb5kdc[22255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.157.131@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 02:43:55 dist-test-slave-kznw krb5kdc[22255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.157.131@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 02:43:55.669661 23320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:55.664282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:45019 (local address 127.13.157.131:48141)
0925 02:43:55.664575 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:55.664589 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:55.664703 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:55.664934 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:55.664937 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:55.665150 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:55.665243 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:55.665250 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:55.665980 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:55.665983 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:55.666616 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:55.666627 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:55.667087 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:55.667090 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:55.667177 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:55.667678 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:55.667689 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:55.669539 (+ 1850us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.157.190:45019: Server oryx/127.13.157.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":95,"threads_started":1}
W20260925 02:43:55.669777 23318 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:43:55.664363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.131:48141 (local address 127.13.157.190:45019)
0925 02:43:55.664568 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:55.664573 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:55.664655 (+ 82us) server_negotiation.cc:407] Connection header received
0925 02:43:55.664789 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:55.664792 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:55.664839 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:55.664902 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:55.665354 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:55.665877 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:55.666716 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:55.666987 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:55.667074 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:55.669675 (+ 2601us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.157.131:48141: BlockingRecv error: recv got EOF from 127.13.157.131:48141 (error 108)
Metrics: {"server-negotiator.queue_time_us":121}
Sep 25 02:43:56 dist-test-slave-kznw krb5kdc[22255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.157.129@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 02:43:56 dist-test-slave-kznw krb5kdc[22255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.157.129@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 02:43:56.326335 23321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:56.316038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:45019 (local address 127.13.157.129:48907)
0925 02:43:56.316298 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:56.316311 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:56.316413 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:56.316624 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:56.316626 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:56.316797 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:56.316890 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:56.316895 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:56.317613 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:56.317616 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:56.318183 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:56.318191 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:56.318608 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:56.318610 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:56.318675 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:56.324162 (+ 5487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:56.324175 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:56.326146 (+ 1971us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.157.190:45019: Server oryx/127.13.157.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":73,"threads_started":1}
W20260925 02:43:56.326416 23322 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:43:56.316082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.129:48907 (local address 127.13.157.190:45019)
0925 02:43:56.316358 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:56.316362 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:56.316374 (+ 12us) server_negotiation.cc:407] Connection header received
0925 02:43:56.316493 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:56.316495 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:56.316538 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:56.316612 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:56.316996 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:56.317529 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:56.318279 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:56.318510 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:56.318585 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:56.326286 (+ 7701us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.157.129:48907: BlockingRecv error: recv got EOF from 127.13.157.129:48907 (error 108)
Metrics: {"server-negotiator.queue_time_us":207,"thread_start_us":83,"threads_started":1}
Sep 25 02:43:56 dist-test-slave-kznw krb5kdc[22255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.157.130@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 02:43:56 dist-test-slave-kznw krb5kdc[22255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.157.130@KRBTEST.COM for oryx/127.13.157.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 02:43:56.491108 23323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:56.485631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:45019 (local address 127.13.157.130:51759)
0925 02:43:56.485927 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:56.485943 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:56.486050 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:56.486301 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:56.486304 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:56.486510 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:56.486610 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:56.486617 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:56.487419 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:56.487423 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:56.488071 (+ 648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:56.488082 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:56.488510 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:56.488513 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:56.488592 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:56.489112 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:56.489124 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:56.490888 (+ 1764us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.157.190:45019: Server oryx/127.13.157.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":84,"threads_started":1}
W20260925 02:43:56.491205 23322 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:43:56.485733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.130:51759 (local address 127.13.157.190:45019)
0925 02:43:56.485927 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:56.485974 (+ 47us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:56.486007 (+ 33us) server_negotiation.cc:407] Connection header received
0925 02:43:56.486147 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:56.486151 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:56.486198 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:56.486261 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:56.486730 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:56.487314 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:56.488184 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:56.488407 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:56.488497 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:56.491101 (+ 2604us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.157.130:51759: BlockingRecv error: recv got EOF from 127.13.157.130:51759 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
I20260925 02:43:56.569463 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 22842
I20260925 02:43:56.574740 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 22996
I20260925 02:43:56.579505 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 23129
I20260925 02:43:56.584487 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 22771
2026-09-25T02:43:56Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17850 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-09-25T02:43:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T02:43:56Z Disabled control of system clock
I20260925 02:43:56.613307 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790304178325258-13942-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790304178325258-13942-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790304178325258-13942-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.157.190:46035
--webserver_interface=127.13.157.190
--webserver_port=0
--builtin_ntp_servers=127.13.157.148:45535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.157.190:46035
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260925 02:43:56.723429 23327 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:43:56.723692 23327 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:43:56.723726 23327 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:43:56.727717 23327 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:43:56.727778 23327 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:43:56.727792 23327 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:43:56.727808 23327 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:43:56.727821 23327 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:43:56.732573 23327 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:45535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790304178325258-13942-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790304178325258-13942-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.157.190:46035
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.157.190:46035
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.157.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23327
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790304178325258-13942-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:43:56.733596 23327 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:43:56.734969 23327 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:43:56.743736 23335 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:56.743816 23333 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:56.743816 23332 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:43:56.744051 23327 server_base.cc:1034] running on GCE node
I20260925 02:43:56.744619 23327 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:43:56.745765 23327 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:43:56.746958 23327 hybrid_clock.cc:648] HybridClock initialized: now 1790304236746948 us; error 73 us; skew 500 ppm
I20260925 02:43:56.748769 23327 webserver.cc:460] Webserver started at http://127.13.157.190:42469/ using document root <none> and password file <none>
I20260925 02:43:56.749321 23327 fs_manager.cc:359] Metadata directory not provided
I20260925 02:43:56.749385 23327 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:43:56.749596 23327 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:43:56.751688 23327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790304178325258-13942-0/minicluster-data/master-0/data/instance:
uuid: "47dd66cc90d5409a871542ace5b87360"
format_stamp: "Formatted at 2026-09-25 02:43:56 on dist-test-slave-kznw"
server_key: "75c7a3c4f944cbd87a2988da4480ba8f"
server_key_iv: "a718c5b682891480018620ef561b0fd6"
server_key_version: "encryptionkey@0"
I20260925 02:43:56.752219 23327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790304178325258-13942-0/minicluster-data/master-0/wal/instance:
uuid: "47dd66cc90d5409a871542ace5b87360"
format_stamp: "Formatted at 2026-09-25 02:43:56 on dist-test-slave-kznw"
server_key: "75c7a3c4f944cbd87a2988da4480ba8f"
server_key_iv: "a718c5b682891480018620ef561b0fd6"
server_key_version: "encryptionkey@0"
I20260925 02:43:56.755901 23327 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.002s
I20260925 02:43:56.758548 23341 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:56.759917 23327 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260925 02:43:56.760058 23327 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790304178325258-13942-0/minicluster-data/master-0/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790304178325258-13942-0/minicluster-data/master-0/wal
uuid: "47dd66cc90d5409a871542ace5b87360"
format_stamp: "Formatted at 2026-09-25 02:43:56 on dist-test-slave-kznw"
server_key: "75c7a3c4f944cbd87a2988da4480ba8f"
server_key_iv: "a718c5b682891480018620ef561b0fd6"
server_key_version: "encryptionkey@0"
I20260925 02:43:56.760164 23327 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790304178325258-13942-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790304178325258-13942-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790304178325258-13942-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:43:56.777688 23327 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:43:56.778364 23327 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:43:56.778548 23327 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:43:56.785107 23327 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.190:46035
I20260925 02:43:56.785130 23393 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.190:46035 every 8 connection(s)
I20260925 02:43:56.786000 23327 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
I20260925 02:43:56.788656 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 23327
I20260925 02:43:56.788772 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790304178325258-13942-0/minicluster-data/master-0/wal/instance
I20260925 02:43:56.789120 13942 external_mini_cluster.cc:1428] Setting key 5fed89eed36ee1f25003a2f06eaa90a5
I20260925 02:43:56.789172 23394 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:43:56.795521 23394 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 47dd66cc90d5409a871542ace5b87360: Bootstrap starting.
I20260925 02:43:56.797740 23394 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 47dd66cc90d5409a871542ace5b87360: Neither blocks nor log segments found. Creating new log.
I20260925 02:43:56.797842 23397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:56.790422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59952 (local address 127.13.157.190:46035)
0925 02:43:56.791048 (+ 626us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:56.791063 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:56.791101 (+ 38us) server_negotiation.cc:407] Connection header received
0925 02:43:56.791755 (+ 654us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:56.791771 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:56.792149 (+ 378us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:56.792467 (+ 318us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:56.792880 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:56.793856 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:56.794709 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:56.795218 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:56.795360 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:56.796251 (+ 891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:56.796255 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:56.796268 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:43:56.796637 (+ 369us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:56.796688 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:43:56.796708 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:56.796790 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:56.796922 (+ 132us) server_negotiation.cc:299] Negotiation successful
0925 02:43:56.797158 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":254,"thread_start_us":103,"threads_started":1}
I20260925 02:43:56.798486 23394 log.cc:824] T 00000000000000000000000000000000 P 47dd66cc90d5409a871542ace5b87360: Log is configured to *not* fsync() on all Append() calls
I20260925 02:43:56.800477 23394 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 47dd66cc90d5409a871542ace5b87360: No bootstrap required, opened a new log
I20260925 02:43:56.804150 23394 raft_consensus.cc:348] T 00000000000000000000000000000000 P 47dd66cc90d5409a871542ace5b87360 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47dd66cc90d5409a871542ace5b87360" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 46035 } }
I20260925 02:43:56.804386 23394 raft_consensus.cc:374] T 00000000000000000000000000000000 P 47dd66cc90d5409a871542ace5b87360 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:43:56.804451 23394 raft_consensus.cc:729] T 00000000000000000000000000000000 P 47dd66cc90d5409a871542ace5b87360 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47dd66cc90d5409a871542ace5b87360, State: Initialized, Role: FOLLOWER
I20260925 02:43:56.804870 23394 consensus_queue.cc:260] T 00000000000000000000000000000000 P 47dd66cc90d5409a871542ace5b87360 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47dd66cc90d5409a871542ace5b87360" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 46035 } }
I20260925 02:43:56.805011 23394 raft_consensus.cc:388] T 00000000000000000000000000000000 P 47dd66cc90d5409a871542ace5b87360 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 02:43:56.805063 23394 raft_consensus.cc:482] T 00000000000000000000000000000000 P 47dd66cc90d5409a871542ace5b87360 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 02:43:56.805135 23394 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 47dd66cc90d5409a871542ace5b87360 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:43:56.806006 23394 raft_consensus.cc:504] T 00000000000000000000000000000000 P 47dd66cc90d5409a871542ace5b87360 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47dd66cc90d5409a871542ace5b87360" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 46035 } }
I20260925 02:43:56.806385 23394 leader_election.cc:302] T 00000000000000000000000000000000 P 47dd66cc90d5409a871542ace5b87360 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 47dd66cc90d5409a871542ace5b87360; no voters:
I20260925 02:43:56.806723 23394 leader_election.cc:288] T 00000000000000000000000000000000 P 47dd66cc90d5409a871542ace5b87360 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 02:43:56.806816 23399 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 47dd66cc90d5409a871542ace5b87360 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:43:56.807044 23399 raft_consensus.cc:686] T 00000000000000000000000000000000 P 47dd66cc90d5409a871542ace5b87360 [term 1 LEADER]: Becoming Leader. State: Replica: 47dd66cc90d5409a871542ace5b87360, State: Running, Role: LEADER
I20260925 02:43:56.807343 23399 consensus_queue.cc:237] T 00000000000000000000000000000000 P 47dd66cc90d5409a871542ace5b87360 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47dd66cc90d5409a871542ace5b87360" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 46035 } }
I20260925 02:43:56.808110 23394 sys_catalog.cc:564] T 00000000000000000000000000000000 P 47dd66cc90d5409a871542ace5b87360 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:43:56.809207 23401 sys_catalog.cc:455] T 00000000000000000000000000000000 P 47dd66cc90d5409a871542ace5b87360 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "47dd66cc90d5409a871542ace5b87360" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47dd66cc90d5409a871542ace5b87360" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 46035 } } }
I20260925 02:43:56.809329 23401 sys_catalog.cc:458] T 00000000000000000000000000000000 P 47dd66cc90d5409a871542ace5b87360 [sys.catalog]: This master's current role is: LEADER
I20260925 02:43:56.809460 23400 sys_catalog.cc:455] T 00000000000000000000000000000000 P 47dd66cc90d5409a871542ace5b87360 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 47dd66cc90d5409a871542ace5b87360. Latest consensus state: current_term: 1 leader_uuid: "47dd66cc90d5409a871542ace5b87360" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47dd66cc90d5409a871542ace5b87360" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 46035 } } }
I20260925 02:43:56.809623 23400 sys_catalog.cc:458] T 00000000000000000000000000000000 P 47dd66cc90d5409a871542ace5b87360 [sys.catalog]: This master's current role is: LEADER
I20260925 02:43:56.809878 23408 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:43:56.812980 23408 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:43:56.818877 23408 catalog_manager.cc:1347] Generated new cluster ID: a82b96dceffb4111832ea4c734dbc317
I20260925 02:43:56.819038 23408 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 02:43:56.833272 23408 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 02:43:56.834250 23408 catalog_manager.cc:1504] Loading token signing keys...
I20260925 02:43:56.837885 23408 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 47dd66cc90d5409a871542ace5b87360: Generated new TSK 0
I20260925 02:43:56.838678 23408 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 02:43:56.852036 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790304178325258-13942-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790304178325258-13942-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790304178325258-13942-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790304178325258-13942-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.157.129:0
--local_ip_for_outbound_sockets=127.13.157.129
--webserver_interface=127.13.157.129
--webserver_port=0
--tserver_master_addrs=127.13.157.190:46035
--builtin_ntp_servers=127.13.157.148:45535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260925 02:43:56.962984 23418 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:43:56.963243 23418 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:43:56.963279 23418 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:43:56.967470 23418 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:43:56.967532 23418 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:43:56.967594 23418 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.157.129
I20260925 02:43:56.972412 23418 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:45535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790304178325258-13942-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790304178325258-13942-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.157.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790304178325258-13942-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.157.129
--webserver_port=0
--tserver_master_addrs=127.13.157.190:46035
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23418
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.157.129
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790304178325258-13942-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:43:56.973546 23418 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:43:56.974992 23418 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:43:56.984400 23424 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:43:56.984458 23418 server_base.cc:1034] running on GCE node
W20260925 02:43:56.984421 23423 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:56.984383 23426 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:43:56.984997 23418 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:43:56.985630 23418 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:43:56.986792 23418 hybrid_clock.cc:648] HybridClock initialized: now 1790304236986777 us; error 24 us; skew 500 ppm
I20260925 02:43:56.988642 23418 webserver.cc:460] Webserver started at http://127.13.157.129:37563/ using document root <none> and password file <none>
I20260925 02:43:56.989192 23418 fs_manager.cc:359] Metadata directory not provided
I20260925 02:43:56.989256 23418 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:43:56.989456 23418 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:43:56.991520 23418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790304178325258-13942-0/minicluster-data/ts-0/data/instance:
uuid: "5852940f6cbe4865846d86984bfcbb3a"
format_stamp: "Formatted at 2026-09-25 02:43:56 on dist-test-slave-kznw"
server_key: "2cceebace99507676cc73248a41ed973"
server_key_iv: "dfd057312fa73892eff98906e95bb062"
server_key_version: "encryptionkey@0"
I20260925 02:43:56.992051 23418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790304178325258-13942-0/minicluster-data/ts-0/wal/instance:
uuid: "5852940f6cbe4865846d86984bfcbb3a"
format_stamp: "Formatted at 2026-09-25 02:43:56 on dist-test-slave-kznw"
server_key: "2cceebace99507676cc73248a41ed973"
server_key_iv: "dfd057312fa73892eff98906e95bb062"
server_key_version: "encryptionkey@0"
I20260925 02:43:56.995617 23418 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260925 02:43:56.998169 23432 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:56.999516 23418 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260925 02:43:56.999635 23418 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790304178325258-13942-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790304178325258-13942-0/minicluster-data/ts-0/wal
uuid: "5852940f6cbe4865846d86984bfcbb3a"
format_stamp: "Formatted at 2026-09-25 02:43:56 on dist-test-slave-kznw"
server_key: "2cceebace99507676cc73248a41ed973"
server_key_iv: "dfd057312fa73892eff98906e95bb062"
server_key_version: "encryptionkey@0"
I20260925 02:43:56.999733 23418 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790304178325258-13942-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790304178325258-13942-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790304178325258-13942-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:43:57.014755 23418 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:43:57.015518 23418 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:43:57.015695 23418 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:43:57.016295 23418 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:43:57.017247 23418 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:43:57.017295 23418 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:57.017349 23418 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:43:57.017375 23418 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:57.025702 23418 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.129:42915
I20260925 02:43:57.025736 23545 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.129:42915 every 8 connection(s)
I20260925 02:43:57.026607 23418 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790304178325258-13942-0/minicluster-data/ts-0/data/info.pb
I20260925 02:43:57.028463 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 23418
I20260925 02:43:57.028586 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790304178325258-13942-0/minicluster-data/ts-0/wal/instance
I20260925 02:43:57.028873 13942 external_mini_cluster.cc:1428] Setting key 06e4c186c3bf2d4d46ed18628e34f359
I20260925 02:43:57.030980 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790304178325258-13942-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790304178325258-13942-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790304178325258-13942-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790304178325258-13942-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.157.130:0
--local_ip_for_outbound_sockets=127.13.157.130
--webserver_interface=127.13.157.130
--webserver_port=0
--tserver_master_addrs=127.13.157.190:46035
--builtin_ntp_servers=127.13.157.148:45535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260925 02:43:57.036473 23397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:57.028531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.129:44637 (local address 127.13.157.190:46035)
0925 02:43:57.028693 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:57.028697 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:57.029854 (+ 1157us) server_negotiation.cc:407] Connection header received
0925 02:43:57.031621 (+ 1767us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:57.031624 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:57.031675 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:57.031737 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:57.032826 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:57.033379 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:57.034592 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:57.034827 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:57.034944 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:57.035995 (+ 1051us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:57.035998 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:57.036000 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:43:57.036012 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:57.036031 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:43:57.036039 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:57.036097 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:57.036238 (+ 141us) server_negotiation.cc:299] Negotiation successful
0925 02:43:57.036319 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260925 02:43:57.037333 23548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:57.028951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:46035 (local address 127.13.157.129:44637)
0925 02:43:57.029703 (+ 752us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:57.029756 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:57.031353 (+ 1597us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:57.031939 (+ 586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:57.031957 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:57.032067 (+ 110us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:57.032689 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:57.032703 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:57.033530 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:57.033534 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:57.034467 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:57.034477 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:57.035002 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:57.035007 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:57.035173 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:57.035728 (+ 555us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 02:43:57.035729 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:57.035773 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 02:43:57.035840 (+ 67us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:43:57.036107 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:43:57.036117 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:43:57.036123 (+ 6us) client_negotiation.cc:769] Sending connection context
0925 02:43:57.036230 (+ 107us) client_negotiation.cc:240] Negotiation successful
0925 02:43:57.036535 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":117,"threads_started":1}
I20260925 02:43:57.038650 23546 heartbeater.cc:347] Connected to a master server at 127.13.157.190:46035
I20260925 02:43:57.038890 23546 heartbeater.cc:464] Registering TS with master...
I20260925 02:43:57.039466 23546 heartbeater.cc:511] Master 127.13.157.190:46035 requested a full tablet report, sending...
I20260925 02:43:57.041039 23358 ts_manager.cc:194] Registered new tserver with Master: 5852940f6cbe4865846d86984bfcbb3a (127.13.157.129:42915)
I20260925 02:43:57.042194 23358 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.157.129:44637
W20260925 02:43:57.141994 23549 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:43:57.142253 23549 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:43:57.142313 23549 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:43:57.146337 23549 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:43:57.146410 23549 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:43:57.146482 23549 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.157.130
I20260925 02:43:57.151249 23549 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:45535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790304178325258-13942-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790304178325258-13942-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.157.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790304178325258-13942-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.157.130
--webserver_port=0
--tserver_master_addrs=127.13.157.190:46035
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23549
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.157.130
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790304178325258-13942-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:43:57.152287 23549 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:43:57.153595 23549 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 02:43:57.163130 23549 server_base.cc:1034] running on GCE node
W20260925 02:43:57.163172 23554 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:57.163050 23557 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:57.163360 23555 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:43:57.163779 23549 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:43:57.164486 23549 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:43:57.165678 23549 hybrid_clock.cc:648] HybridClock initialized: now 1790304237165664 us; error 35 us; skew 500 ppm
I20260925 02:43:57.167661 23549 webserver.cc:460] Webserver started at http://127.13.157.130:44745/ using document root <none> and password file <none>
I20260925 02:43:57.168215 23549 fs_manager.cc:359] Metadata directory not provided
I20260925 02:43:57.168284 23549 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:43:57.168491 23549 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:43:57.170555 23549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790304178325258-13942-0/minicluster-data/ts-1/data/instance:
uuid: "827a17037a2e4b4bb0d1019eac7be8a8"
format_stamp: "Formatted at 2026-09-25 02:43:57 on dist-test-slave-kznw"
server_key: "3a3620092e737bbde77840807e890731"
server_key_iv: "885431b2226294ae9386318aa2461801"
server_key_version: "encryptionkey@0"
I20260925 02:43:57.171137 23549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790304178325258-13942-0/minicluster-data/ts-1/wal/instance:
uuid: "827a17037a2e4b4bb0d1019eac7be8a8"
format_stamp: "Formatted at 2026-09-25 02:43:57 on dist-test-slave-kznw"
server_key: "3a3620092e737bbde77840807e890731"
server_key_iv: "885431b2226294ae9386318aa2461801"
server_key_version: "encryptionkey@0"
I20260925 02:43:57.174788 23549 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.003s
I20260925 02:43:57.177387 23563 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:57.178679 23549 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260925 02:43:57.178793 23549 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790304178325258-13942-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790304178325258-13942-0/minicluster-data/ts-1/wal
uuid: "827a17037a2e4b4bb0d1019eac7be8a8"
format_stamp: "Formatted at 2026-09-25 02:43:57 on dist-test-slave-kznw"
server_key: "3a3620092e737bbde77840807e890731"
server_key_iv: "885431b2226294ae9386318aa2461801"
server_key_version: "encryptionkey@0"
I20260925 02:43:57.178925 23549 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790304178325258-13942-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790304178325258-13942-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790304178325258-13942-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:43:57.188089 23549 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:43:57.188740 23549 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:43:57.188941 23549 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:43:57.189509 23549 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:43:57.190429 23549 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:43:57.190485 23549 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:57.190537 23549 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:43:57.190565 23549 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:57.198889 23549 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.130:43027
I20260925 02:43:57.199002 23676 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.130:43027 every 8 connection(s)
I20260925 02:43:57.199824 23549 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790304178325258-13942-0/minicluster-data/ts-1/data/info.pb
I20260925 02:43:57.207561 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 23549
I20260925 02:43:57.207698 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790304178325258-13942-0/minicluster-data/ts-1/wal/instance
I20260925 02:43:57.208056 13942 external_mini_cluster.cc:1428] Setting key 101c0a2304595197cd526aaa54a32d1b
I20260925 02:43:57.209544 23397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:57.201703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.130:36073 (local address 127.13.157.190:46035)
0925 02:43:57.201845 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:57.201848 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:57.202999 (+ 1151us) server_negotiation.cc:407] Connection header received
0925 02:43:57.204428 (+ 1429us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:57.204432 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:57.204483 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:57.204559 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:57.205531 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:57.206108 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:57.207376 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:57.207604 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:57.207698 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:57.209004 (+ 1306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:57.209009 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:57.209012 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:43:57.209028 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:57.209054 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:43:57.209064 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:57.209133 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:57.209295 (+ 162us) server_negotiation.cc:299] Negotiation successful
0925 02:43:57.209381 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260925 02:43:57.210127 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790304178325258-13942-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790304178325258-13942-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790304178325258-13942-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790304178325258-13942-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.157.131:0
--local_ip_for_outbound_sockets=127.13.157.131
--webserver_interface=127.13.157.131
--webserver_port=0
--tserver_master_addrs=127.13.157.190:46035
--builtin_ntp_servers=127.13.157.148:45535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260925 02:43:57.210628 23679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:57.202124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:46035 (local address 127.13.157.130:36073)
0925 02:43:57.202822 (+ 698us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:57.202864 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:57.204205 (+ 1341us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:57.204764 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:57.204772 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:57.204855 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:57.205360 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:57.205374 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:57.206235 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:57.206238 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:57.207234 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:57.207244 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:57.207753 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:57.207757 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:57.207965 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:57.208646 (+ 681us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 02:43:57.208647 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:57.208693 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 02:43:57.208763 (+ 70us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:43:57.209146 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:43:57.209157 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:43:57.209164 (+ 7us) client_negotiation.cc:769] Sending connection context
0925 02:43:57.209288 (+ 124us) client_negotiation.cc:240] Negotiation successful
0925 02:43:57.209653 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":120,"threads_started":1}
I20260925 02:43:57.211853 23677 heartbeater.cc:347] Connected to a master server at 127.13.157.190:46035
I20260925 02:43:57.212092 23677 heartbeater.cc:464] Registering TS with master...
I20260925 02:43:57.212620 23677 heartbeater.cc:511] Master 127.13.157.190:46035 requested a full tablet report, sending...
I20260925 02:43:57.213644 23358 ts_manager.cc:194] Registered new tserver with Master: 827a17037a2e4b4bb0d1019eac7be8a8 (127.13.157.130:43027)
I20260925 02:43:57.214200 23358 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.157.130:36073
W20260925 02:43:57.319078 23680 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:43:57.319342 23680 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:43:57.319396 23680 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:43:57.323443 23680 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:43:57.323505 23680 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:43:57.323578 23680 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.157.131
I20260925 02:43:57.328341 23680 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:45535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790304178325258-13942-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790304178325258-13942-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.157.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790304178325258-13942-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.157.131
--webserver_port=0
--tserver_master_addrs=127.13.157.190:46035
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23680
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.157.131
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790304178325258-13942-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:43:57.329419 23680 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:43:57.330772 23680 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:43:57.340420 23688 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:57.340420 23685 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:43:57.340562 23680 server_base.cc:1034] running on GCE node
W20260925 02:43:57.340781 23686 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:43:57.341253 23680 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:43:57.341905 23680 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:43:57.343065 23680 hybrid_clock.cc:648] HybridClock initialized: now 1790304237343042 us; error 34 us; skew 500 ppm
I20260925 02:43:57.345041 23680 webserver.cc:460] Webserver started at http://127.13.157.131:34439/ using document root <none> and password file <none>
I20260925 02:43:57.345626 23680 fs_manager.cc:359] Metadata directory not provided
I20260925 02:43:57.345700 23680 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:43:57.345939 23680 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:43:57.348138 23680 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790304178325258-13942-0/minicluster-data/ts-2/data/instance:
uuid: "4cc170f4dae141ad8be22f095f805dea"
format_stamp: "Formatted at 2026-09-25 02:43:57 on dist-test-slave-kznw"
server_key: "0b5b5b33caf75c90e7607e6d709ca768"
server_key_iv: "d4fee54822752252cd6b4c683fd73bec"
server_key_version: "encryptionkey@0"
I20260925 02:43:57.348709 23680 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790304178325258-13942-0/minicluster-data/ts-2/wal/instance:
uuid: "4cc170f4dae141ad8be22f095f805dea"
format_stamp: "Formatted at 2026-09-25 02:43:57 on dist-test-slave-kznw"
server_key: "0b5b5b33caf75c90e7607e6d709ca768"
server_key_iv: "d4fee54822752252cd6b4c683fd73bec"
server_key_version: "encryptionkey@0"
I20260925 02:43:57.352489 23680 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260925 02:43:57.355091 23694 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:57.356321 23680 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260925 02:43:57.356447 23680 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790304178325258-13942-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790304178325258-13942-0/minicluster-data/ts-2/wal
uuid: "4cc170f4dae141ad8be22f095f805dea"
format_stamp: "Formatted at 2026-09-25 02:43:57 on dist-test-slave-kznw"
server_key: "0b5b5b33caf75c90e7607e6d709ca768"
server_key_iv: "d4fee54822752252cd6b4c683fd73bec"
server_key_version: "encryptionkey@0"
I20260925 02:43:57.356556 23680 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790304178325258-13942-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790304178325258-13942-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790304178325258-13942-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:43:57.382426 23680 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:43:57.383244 23680 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:43:57.383435 23680 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:43:57.384039 23680 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:43:57.384991 23680 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:43:57.385049 23680 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:57.385104 23680 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:43:57.385129 23680 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:57.394654 23680 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.131:45937
I20260925 02:43:57.394683 23807 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.131:45937 every 8 connection(s)
I20260925 02:43:57.395586 23680 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790304178325258-13942-0/minicluster-data/ts-2/data/info.pb
I20260925 02:43:57.396677 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 23680
I20260925 02:43:57.396781 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790304178325258-13942-0/minicluster-data/ts-2/wal/instance
I20260925 02:43:57.397091 13942 external_mini_cluster.cc:1428] Setting key 21717119e0dd76bacd4a54475ab68d42
I20260925 02:43:57.404600 23397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:57.397493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.131:35811 (local address 127.13.157.190:46035)
0925 02:43:57.397630 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:57.397634 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:57.398883 (+ 1249us) server_negotiation.cc:407] Connection header received
0925 02:43:57.400275 (+ 1392us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:57.400277 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:57.400322 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:57.400389 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:57.401204 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:57.401723 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:57.402816 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:57.403119 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:57.403195 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:57.404237 (+ 1042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:57.404241 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:57.404244 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:43:57.404258 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:57.404276 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:43:57.404284 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:57.404339 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:57.404439 (+ 100us) server_negotiation.cc:299] Negotiation successful
0925 02:43:57.404488 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260925 02:43:57.405390 23810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:57.398029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:46035 (local address 127.13.157.131:35811)
0925 02:43:57.398730 (+ 701us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:57.398780 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:57.400086 (+ 1306us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:57.400516 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:57.400522 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:57.400604 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:57.401081 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:57.401092 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:57.401810 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:57.401812 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:57.402713 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:57.402722 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:57.403252 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:57.403255 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:57.403395 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:57.403899 (+ 504us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 02:43:57.403900 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:57.403973 (+ 73us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 02:43:57.404043 (+ 70us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:43:57.404340 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:43:57.404344 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:43:57.404348 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 02:43:57.404429 (+ 81us) client_negotiation.cc:240] Negotiation successful
0925 02:43:57.404635 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":112,"threads_started":1}
I20260925 02:43:57.406474 23808 heartbeater.cc:347] Connected to a master server at 127.13.157.190:46035
I20260925 02:43:57.406682 23808 heartbeater.cc:464] Registering TS with master...
I20260925 02:43:57.407191 23808 heartbeater.cc:511] Master 127.13.157.190:46035 requested a full tablet report, sending...
I20260925 02:43:57.408128 23358 ts_manager.cc:194] Registered new tserver with Master: 4cc170f4dae141ad8be22f095f805dea (127.13.157.131:45937)
I20260925 02:43:57.408702 23358 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.157.131:35811
I20260925 02:43:57.410732 13942 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260925 02:43:57.413605 23397 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:43:57.412623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59968 (local address 127.13.157.190:46035)
0925 02:43:57.412757 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:57.412760 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:57.412890 (+ 130us) server_negotiation.cc:407] Connection header received
0925 02:43:57.412966 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:57.412969 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:57.413017 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:57.413084 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:57.413493 (+ 409us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59968: BlockingRecv error: recv got EOF from 127.0.0.1:59968 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
I20260925 02:43:57.413720 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 23418
I20260925 02:43:57.419380 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 23549
I20260925 02:43:57.424186 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 23680
I20260925 02:43:57.428911 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 23327
2026-09-25T02:43:57Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (842 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-09-25T02:43:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T02:43:57Z Disabled control of system clock
I20260925 02:43:57.455613 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790304178325258-13942-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790304178325258-13942-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790304178325258-13942-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.157.190:39553
--webserver_interface=127.13.157.190
--webserver_port=0
--builtin_ntp_servers=127.13.157.148:39497
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.157.190:39553
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260925 02:43:57.563086 23819 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:43:57.563347 23819 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:43:57.563386 23819 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:43:57.567353 23819 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:43:57.567401 23819 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:43:57.567415 23819 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:43:57.567430 23819 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:43:57.567443 23819 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:43:57.572192 23819 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:39497
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790304178325258-13942-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790304178325258-13942-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.157.190:39553
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.157.190:39553
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.157.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23819
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790304178325258-13942-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:43:57.573180 23819 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:43:57.574486 23819 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:43:57.582706 23825 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:57.582712 23827 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:57.582712 23824 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:43:57.583103 23819 server_base.cc:1034] running on GCE node
I20260925 02:43:57.583694 23819 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:43:57.584769 23819 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:43:57.585958 23819 hybrid_clock.cc:648] HybridClock initialized: now 1790304237585945 us; error 28 us; skew 500 ppm
I20260925 02:43:57.587910 23819 webserver.cc:460] Webserver started at http://127.13.157.190:34853/ using document root <none> and password file <none>
I20260925 02:43:57.588465 23819 fs_manager.cc:359] Metadata directory not provided
I20260925 02:43:57.588536 23819 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:43:57.588733 23819 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:43:57.590842 23819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790304178325258-13942-0/minicluster-data/master-0/data/instance:
uuid: "f42d5c7881814f4f8d0a2eb9d4fd0e60"
format_stamp: "Formatted at 2026-09-25 02:43:57 on dist-test-slave-kznw"
server_key: "0e51fb945828882968984e30788af6e4"
server_key_iv: "4ea11409c6fd395fc3c0a7834ded8e5a"
server_key_version: "encryptionkey@0"
I20260925 02:43:57.591418 23819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790304178325258-13942-0/minicluster-data/master-0/wal/instance:
uuid: "f42d5c7881814f4f8d0a2eb9d4fd0e60"
format_stamp: "Formatted at 2026-09-25 02:43:57 on dist-test-slave-kznw"
server_key: "0e51fb945828882968984e30788af6e4"
server_key_iv: "4ea11409c6fd395fc3c0a7834ded8e5a"
server_key_version: "encryptionkey@0"
I20260925 02:43:57.595019 23819 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260925 02:43:57.597317 23833 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:57.598582 23819 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260925 02:43:57.598695 23819 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790304178325258-13942-0/minicluster-data/master-0/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790304178325258-13942-0/minicluster-data/master-0/wal
uuid: "f42d5c7881814f4f8d0a2eb9d4fd0e60"
format_stamp: "Formatted at 2026-09-25 02:43:57 on dist-test-slave-kznw"
server_key: "0e51fb945828882968984e30788af6e4"
server_key_iv: "4ea11409c6fd395fc3c0a7834ded8e5a"
server_key_version: "encryptionkey@0"
I20260925 02:43:57.598803 23819 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790304178325258-13942-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790304178325258-13942-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790304178325258-13942-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:43:57.603397 23819 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:43:57.604013 23819 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:43:57.604174 23819 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:43:57.610846 23819 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.190:39553
I20260925 02:43:57.610867 23886 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.190:39553 every 8 connection(s)
I20260925 02:43:57.611883 23819 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
I20260925 02:43:57.615303 23887 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:43:57.620800 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 23819
I20260925 02:43:57.620910 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790304178325258-13942-0/minicluster-data/master-0/wal/instance
I20260925 02:43:57.621045 23887 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f42d5c7881814f4f8d0a2eb9d4fd0e60: Bootstrap starting.
I20260925 02:43:57.621184 13942 external_mini_cluster.cc:1428] Setting key 247bd1be7202a20342b2641a52a0dcce
I20260925 02:43:57.623639 23887 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f42d5c7881814f4f8d0a2eb9d4fd0e60: Neither blocks nor log segments found. Creating new log.
I20260925 02:43:57.624364 23887 log.cc:824] T 00000000000000000000000000000000 P f42d5c7881814f4f8d0a2eb9d4fd0e60: Log is configured to *not* fsync() on all Append() calls
I20260925 02:43:57.626519 23887 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f42d5c7881814f4f8d0a2eb9d4fd0e60: No bootstrap required, opened a new log
I20260925 02:43:57.626668 23890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:57.622420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59042 (local address 127.13.157.190:39553)
0925 02:43:57.623090 (+ 670us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:57.623107 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:57.623145 (+ 38us) server_negotiation.cc:407] Connection header received
0925 02:43:57.623810 (+ 665us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:57.623829 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:57.624197 (+ 368us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:57.624480 (+ 283us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:57.625251 (+ 771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:57.625257 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:57.625272 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:43:57.625284 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:57.625357 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:43:57.625379 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:57.625479 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:57.625552 (+ 73us) server_negotiation.cc:299] Negotiation successful
0925 02:43:57.625791 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":267,"thread_start_us":120,"threads_started":1}
I20260925 02:43:57.630740 23887 raft_consensus.cc:348] T 00000000000000000000000000000000 P f42d5c7881814f4f8d0a2eb9d4fd0e60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f42d5c7881814f4f8d0a2eb9d4fd0e60" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 39553 } }
I20260925 02:43:57.630995 23887 raft_consensus.cc:374] T 00000000000000000000000000000000 P f42d5c7881814f4f8d0a2eb9d4fd0e60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:43:57.631052 23887 raft_consensus.cc:729] T 00000000000000000000000000000000 P f42d5c7881814f4f8d0a2eb9d4fd0e60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f42d5c7881814f4f8d0a2eb9d4fd0e60, State: Initialized, Role: FOLLOWER
I20260925 02:43:57.631407 23887 consensus_queue.cc:260] T 00000000000000000000000000000000 P f42d5c7881814f4f8d0a2eb9d4fd0e60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f42d5c7881814f4f8d0a2eb9d4fd0e60" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 39553 } }
I20260925 02:43:57.631515 23887 raft_consensus.cc:388] T 00000000000000000000000000000000 P f42d5c7881814f4f8d0a2eb9d4fd0e60 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 02:43:57.631565 23887 raft_consensus.cc:482] T 00000000000000000000000000000000 P f42d5c7881814f4f8d0a2eb9d4fd0e60 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 02:43:57.631633 23887 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f42d5c7881814f4f8d0a2eb9d4fd0e60 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:43:57.632568 23887 raft_consensus.cc:504] T 00000000000000000000000000000000 P f42d5c7881814f4f8d0a2eb9d4fd0e60 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f42d5c7881814f4f8d0a2eb9d4fd0e60" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 39553 } }
I20260925 02:43:57.632894 23887 leader_election.cc:302] T 00000000000000000000000000000000 P f42d5c7881814f4f8d0a2eb9d4fd0e60 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f42d5c7881814f4f8d0a2eb9d4fd0e60; no voters:
I20260925 02:43:57.633142 23887 leader_election.cc:288] T 00000000000000000000000000000000 P f42d5c7881814f4f8d0a2eb9d4fd0e60 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 02:43:57.633234 23892 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f42d5c7881814f4f8d0a2eb9d4fd0e60 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:43:57.633469 23892 raft_consensus.cc:686] T 00000000000000000000000000000000 P f42d5c7881814f4f8d0a2eb9d4fd0e60 [term 1 LEADER]: Becoming Leader. State: Replica: f42d5c7881814f4f8d0a2eb9d4fd0e60, State: Running, Role: LEADER
I20260925 02:43:57.633764 23892 consensus_queue.cc:237] T 00000000000000000000000000000000 P f42d5c7881814f4f8d0a2eb9d4fd0e60 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f42d5c7881814f4f8d0a2eb9d4fd0e60" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 39553 } }
I20260925 02:43:57.634251 23887 sys_catalog.cc:564] T 00000000000000000000000000000000 P f42d5c7881814f4f8d0a2eb9d4fd0e60 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:43:57.635248 23893 sys_catalog.cc:455] T 00000000000000000000000000000000 P f42d5c7881814f4f8d0a2eb9d4fd0e60 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f42d5c7881814f4f8d0a2eb9d4fd0e60" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f42d5c7881814f4f8d0a2eb9d4fd0e60" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 39553 } } }
I20260925 02:43:57.635367 23893 sys_catalog.cc:458] T 00000000000000000000000000000000 P f42d5c7881814f4f8d0a2eb9d4fd0e60 [sys.catalog]: This master's current role is: LEADER
I20260925 02:43:57.635761 23894 sys_catalog.cc:455] T 00000000000000000000000000000000 P f42d5c7881814f4f8d0a2eb9d4fd0e60 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f42d5c7881814f4f8d0a2eb9d4fd0e60. Latest consensus state: current_term: 1 leader_uuid: "f42d5c7881814f4f8d0a2eb9d4fd0e60" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f42d5c7881814f4f8d0a2eb9d4fd0e60" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 39553 } } }
I20260925 02:43:57.635880 23894 sys_catalog.cc:458] T 00000000000000000000000000000000 P f42d5c7881814f4f8d0a2eb9d4fd0e60 [sys.catalog]: This master's current role is: LEADER
I20260925 02:43:57.635834 23901 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:43:57.638552 23901 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:43:57.643754 23901 catalog_manager.cc:1347] Generated new cluster ID: 3e4433204c5c4043b3e75d23c0d8a167
I20260925 02:43:57.643826 23901 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 02:43:57.693941 23901 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 02:43:57.694262 23901 catalog_manager.cc:1504] Loading token signing keys...
I20260925 02:43:57.700870 23901 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f42d5c7881814f4f8d0a2eb9d4fd0e60: Generated new TSK 0
I20260925 02:43:57.701591 23901 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 02:43:57.706780 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790304178325258-13942-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790304178325258-13942-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790304178325258-13942-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790304178325258-13942-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.157.129:0
--local_ip_for_outbound_sockets=127.13.157.129
--webserver_interface=127.13.157.129
--webserver_port=0
--tserver_master_addrs=127.13.157.190:39553
--builtin_ntp_servers=127.13.157.148:39497
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260925 02:43:57.823721 23911 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:43:57.823978 23911 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:43:57.824018 23911 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:43:57.828025 23911 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:43:57.828080 23911 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:43:57.828137 23911 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.157.129
I20260925 02:43:57.832954 23911 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:39497
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790304178325258-13942-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790304178325258-13942-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.157.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790304178325258-13942-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.157.129
--webserver_port=0
--tserver_master_addrs=127.13.157.190:39553
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23911
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.157.129
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790304178325258-13942-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:43:57.834053 23911 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:43:57.835553 23911 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:43:57.845216 23920 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:57.845319 23918 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:43:57.845445 23911 server_base.cc:1034] running on GCE node
W20260925 02:43:57.845216 23917 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:43:57.845890 23911 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:43:57.846625 23911 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:43:57.847784 23911 hybrid_clock.cc:648] HybridClock initialized: now 1790304237847762 us; error 32 us; skew 500 ppm
I20260925 02:43:57.849663 23911 webserver.cc:460] Webserver started at http://127.13.157.129:43459/ using document root <none> and password file <none>
I20260925 02:43:57.850234 23911 fs_manager.cc:359] Metadata directory not provided
I20260925 02:43:57.850301 23911 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:43:57.850517 23911 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:43:57.852854 23911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790304178325258-13942-0/minicluster-data/ts-0/data/instance:
uuid: "1c9833b1bc514ddc86b67e1107bbcdc2"
format_stamp: "Formatted at 2026-09-25 02:43:57 on dist-test-slave-kznw"
server_key: "78df81b1fc5e27cb9da3cd119a0ed691"
server_key_iv: "f84c7cca9066a271b3e354cbba5b3104"
server_key_version: "encryptionkey@0"
I20260925 02:43:57.853442 23911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790304178325258-13942-0/minicluster-data/ts-0/wal/instance:
uuid: "1c9833b1bc514ddc86b67e1107bbcdc2"
format_stamp: "Formatted at 2026-09-25 02:43:57 on dist-test-slave-kznw"
server_key: "78df81b1fc5e27cb9da3cd119a0ed691"
server_key_iv: "f84c7cca9066a271b3e354cbba5b3104"
server_key_version: "encryptionkey@0"
I20260925 02:43:57.857286 23911 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.002s
I20260925 02:43:57.860013 23926 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:57.861382 23911 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 02:43:57.861495 23911 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790304178325258-13942-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790304178325258-13942-0/minicluster-data/ts-0/wal
uuid: "1c9833b1bc514ddc86b67e1107bbcdc2"
format_stamp: "Formatted at 2026-09-25 02:43:57 on dist-test-slave-kznw"
server_key: "78df81b1fc5e27cb9da3cd119a0ed691"
server_key_iv: "f84c7cca9066a271b3e354cbba5b3104"
server_key_version: "encryptionkey@0"
I20260925 02:43:57.861611 23911 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790304178325258-13942-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790304178325258-13942-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790304178325258-13942-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:43:57.866215 23911 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:43:57.866815 23911 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:43:57.867003 23911 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:43:57.867551 23911 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:43:57.868455 23911 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:43:57.868521 23911 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:57.868571 23911 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:43:57.868597 23911 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:57.877143 23911 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.129:45421
I20260925 02:43:57.877177 24040 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.129:45421 every 8 connection(s)
I20260925 02:43:57.878070 23911 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790304178325258-13942-0/minicluster-data/ts-0/data/info.pb
I20260925 02:43:57.883535 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 23911
I20260925 02:43:57.883678 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790304178325258-13942-0/minicluster-data/ts-0/wal/instance
I20260925 02:43:57.883975 13942 external_mini_cluster.cc:1428] Setting key 52f5ab9bd6740de1b789e73bb024fcbb
I20260925 02:43:57.884567 23890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:57.879948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.129:35031 (local address 127.13.157.190:39553)
0925 02:43:57.880120 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:57.880124 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:57.881317 (+ 1193us) server_negotiation.cc:407] Connection header received
0925 02:43:57.882632 (+ 1315us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:57.882636 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:57.882686 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:57.882762 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:57.884087 (+ 1325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:57.884091 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:57.884094 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:43:57.884109 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:57.884134 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:43:57.884142 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:57.884231 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:57.884365 (+ 134us) server_negotiation.cc:299] Negotiation successful
0925 02:43:57.884415 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260925 02:43:57.885651 24043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:57.880370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:39553 (local address 127.13.157.129:35031)
0925 02:43:57.881143 (+ 773us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:57.881187 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:57.882396 (+ 1209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:57.882987 (+ 591us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:57.882996 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:57.883094 (+ 98us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:57.883767 (+ 673us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 02:43:57.883769 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:57.883817 (+ 48us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 02:43:57.883892 (+ 75us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:43:57.884221 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:43:57.884231 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:43:57.884237 (+ 6us) client_negotiation.cc:769] Sending connection context
0925 02:43:57.884411 (+ 174us) client_negotiation.cc:240] Negotiation successful
0925 02:43:57.884703 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":128,"threads_started":1}
I20260925 02:43:57.886174 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790304178325258-13942-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790304178325258-13942-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790304178325258-13942-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790304178325258-13942-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.157.130:0
--local_ip_for_outbound_sockets=127.13.157.130
--webserver_interface=127.13.157.130
--webserver_port=0
--tserver_master_addrs=127.13.157.190:39553
--builtin_ntp_servers=127.13.157.148:39497
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260925 02:43:57.887017 24041 heartbeater.cc:347] Connected to a master server at 127.13.157.190:39553
I20260925 02:43:57.887244 24041 heartbeater.cc:464] Registering TS with master...
I20260925 02:43:57.887668 24041 heartbeater.cc:511] Master 127.13.157.190:39553 requested a full tablet report, sending...
I20260925 02:43:57.889168 23851 ts_manager.cc:194] Registered new tserver with Master: 1c9833b1bc514ddc86b67e1107bbcdc2 (127.13.157.129:45421)
W20260925 02:43:57.998929 24044 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:43:57.999202 24044 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:43:57.999259 24044 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:43:58.003289 24044 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:43:58.003338 24044 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:43:58.003415 24044 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.157.130
I20260925 02:43:58.009016 24044 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:39497
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790304178325258-13942-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790304178325258-13942-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.157.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790304178325258-13942-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.157.130
--webserver_port=0
--tserver_master_addrs=127.13.157.190:39553
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24044
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.157.130
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790304178325258-13942-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:43:58.010421 24044 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:43:58.012151 24044 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:43:58.022480 24050 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:58.022504 24052 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:58.022603 24049 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:43:58.022790 24044 server_base.cc:1034] running on GCE node
I20260925 02:43:58.023257 24044 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:43:58.023914 24044 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:43:58.025084 24044 hybrid_clock.cc:648] HybridClock initialized: now 1790304238025063 us; error 31 us; skew 500 ppm
I20260925 02:43:58.026932 24044 webserver.cc:460] Webserver started at http://127.13.157.130:42321/ using document root <none> and password file <none>
I20260925 02:43:58.027509 24044 fs_manager.cc:359] Metadata directory not provided
I20260925 02:43:58.027572 24044 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:43:58.027771 24044 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:43:58.029852 24044 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790304178325258-13942-0/minicluster-data/ts-1/data/instance:
uuid: "ca723c5b451c4cfaa6fb5e784adf93bd"
format_stamp: "Formatted at 2026-09-25 02:43:58 on dist-test-slave-kznw"
server_key: "5452239e5c6d04f84a06963282f6b677"
server_key_iv: "8479bc51efa74380062912b217b0bc18"
server_key_version: "encryptionkey@0"
I20260925 02:43:58.030421 24044 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790304178325258-13942-0/minicluster-data/ts-1/wal/instance:
uuid: "ca723c5b451c4cfaa6fb5e784adf93bd"
format_stamp: "Formatted at 2026-09-25 02:43:58 on dist-test-slave-kznw"
server_key: "5452239e5c6d04f84a06963282f6b677"
server_key_iv: "8479bc51efa74380062912b217b0bc18"
server_key_version: "encryptionkey@0"
I20260925 02:43:58.034133 24044 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260925 02:43:58.036918 24058 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:58.038169 24044 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260925 02:43:58.038272 24044 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790304178325258-13942-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790304178325258-13942-0/minicluster-data/ts-1/wal
uuid: "ca723c5b451c4cfaa6fb5e784adf93bd"
format_stamp: "Formatted at 2026-09-25 02:43:58 on dist-test-slave-kznw"
server_key: "5452239e5c6d04f84a06963282f6b677"
server_key_iv: "8479bc51efa74380062912b217b0bc18"
server_key_version: "encryptionkey@0"
I20260925 02:43:58.038388 24044 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790304178325258-13942-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790304178325258-13942-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790304178325258-13942-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:43:58.043066 24044 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:43:58.043704 24044 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:43:58.043860 24044 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:43:58.044400 24044 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:43:58.045324 24044 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:43:58.045382 24044 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:58.045420 24044 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:43:58.045447 24044 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:58.053846 24044 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.130:45145
I20260925 02:43:58.053869 24172 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.130:45145 every 8 connection(s)
I20260925 02:43:58.054734 24044 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790304178325258-13942-0/minicluster-data/ts-1/data/info.pb
I20260925 02:43:58.060635 23890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:58.056567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.130:57877 (local address 127.13.157.190:39553)
0925 02:43:58.056742 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:58.056747 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:58.057825 (+ 1078us) server_negotiation.cc:407] Connection header received
0925 02:43:58.059134 (+ 1309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:58.059137 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:58.059176 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:58.059249 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:58.060275 (+ 1026us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:58.060279 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:58.060281 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:43:58.060293 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:58.060313 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:43:58.060321 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:58.060391 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:58.060483 (+ 92us) server_negotiation.cc:299] Negotiation successful
0925 02:43:58.060523 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260925 02:43:58.061506 24175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:58.057010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:39553 (local address 127.13.157.130:57877)
0925 02:43:58.057675 (+ 665us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:58.057718 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:58.058847 (+ 1129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:58.059400 (+ 553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:58.059406 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:58.059494 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:58.060036 (+ 542us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 02:43:58.060038 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:58.060077 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 02:43:58.060142 (+ 65us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:43:58.060380 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:43:58.060385 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:43:58.060388 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 02:43:58.060510 (+ 122us) client_negotiation.cc:240] Negotiation successful
0925 02:43:58.060703 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":103,"threads_started":1}
I20260925 02:43:58.062510 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 24044
I20260925 02:43:58.062626 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790304178325258-13942-0/minicluster-data/ts-1/wal/instance
I20260925 02:43:58.062652 24173 heartbeater.cc:347] Connected to a master server at 127.13.157.190:39553
I20260925 02:43:58.062882 24173 heartbeater.cc:464] Registering TS with master...
I20260925 02:43:58.063004 13942 external_mini_cluster.cc:1428] Setting key 7e7809b476472ed2602cbc18a8dc9c5d
I20260925 02:43:58.063388 24173 heartbeater.cc:511] Master 127.13.157.190:39553 requested a full tablet report, sending...
I20260925 02:43:58.064397 23851 ts_manager.cc:194] Registered new tserver with Master: ca723c5b451c4cfaa6fb5e784adf93bd (127.13.157.130:45145)
I20260925 02:43:58.065165 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790304178325258-13942-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790304178325258-13942-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790304178325258-13942-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790304178325258-13942-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.157.131:0
--local_ip_for_outbound_sockets=127.13.157.131
--webserver_interface=127.13.157.131
--webserver_port=0
--tserver_master_addrs=127.13.157.190:39553
--builtin_ntp_servers=127.13.157.148:39497
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260925 02:43:58.177971 24176 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:43:58.178241 24176 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:43:58.178295 24176 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:43:58.182379 24176 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:43:58.182435 24176 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:43:58.182515 24176 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.157.131
I20260925 02:43:58.187341 24176 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:39497
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790304178325258-13942-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790304178325258-13942-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.157.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790304178325258-13942-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.157.131
--webserver_port=0
--tserver_master_addrs=127.13.157.190:39553
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24176
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.157.131
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790304178325258-13942-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:43:58.188460 24176 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:43:58.189906 24176 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:43:58.199536 24181 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:58.199692 24182 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:58.199536 24184 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:43:58.199767 24176 server_base.cc:1034] running on GCE node
I20260925 02:43:58.200321 24176 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:43:58.201102 24176 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:43:58.202214 24176 hybrid_clock.cc:648] HybridClock initialized: now 1790304238202183 us; error 43 us; skew 500 ppm
I20260925 02:43:58.204139 24176 webserver.cc:460] Webserver started at http://127.13.157.131:37835/ using document root <none> and password file <none>
I20260925 02:43:58.204735 24176 fs_manager.cc:359] Metadata directory not provided
I20260925 02:43:58.204806 24176 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:43:58.205003 24176 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:43:58.207185 24176 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790304178325258-13942-0/minicluster-data/ts-2/data/instance:
uuid: "98c79c2e6d74468ca4e900f1dee1389d"
format_stamp: "Formatted at 2026-09-25 02:43:58 on dist-test-slave-kznw"
server_key: "f0f15f6b5e146a6a0b39e2c1cd1da4ff"
server_key_iv: "139fb964ac9e0aeca342b82fa334dbf1"
server_key_version: "encryptionkey@0"
I20260925 02:43:58.207772 24176 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790304178325258-13942-0/minicluster-data/ts-2/wal/instance:
uuid: "98c79c2e6d74468ca4e900f1dee1389d"
format_stamp: "Formatted at 2026-09-25 02:43:58 on dist-test-slave-kznw"
server_key: "f0f15f6b5e146a6a0b39e2c1cd1da4ff"
server_key_iv: "139fb964ac9e0aeca342b82fa334dbf1"
server_key_version: "encryptionkey@0"
I20260925 02:43:58.211602 24176 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260925 02:43:58.214465 24190 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:58.215902 24176 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 02:43:58.216084 24176 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790304178325258-13942-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790304178325258-13942-0/minicluster-data/ts-2/wal
uuid: "98c79c2e6d74468ca4e900f1dee1389d"
format_stamp: "Formatted at 2026-09-25 02:43:58 on dist-test-slave-kznw"
server_key: "f0f15f6b5e146a6a0b39e2c1cd1da4ff"
server_key_iv: "139fb964ac9e0aeca342b82fa334dbf1"
server_key_version: "encryptionkey@0"
I20260925 02:43:58.216215 24176 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790304178325258-13942-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790304178325258-13942-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790304178325258-13942-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:43:58.221101 24176 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:43:58.221715 24176 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:43:58.221891 24176 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:43:58.222462 24176 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:43:58.223472 24176 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:43:58.223526 24176 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:58.223580 24176 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:43:58.223613 24176 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:58.233165 24176 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.131:40291
I20260925 02:43:58.233233 24304 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.131:40291 every 8 connection(s)
I20260925 02:43:58.234416 24176 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790304178325258-13942-0/minicluster-data/ts-2/data/info.pb
I20260925 02:43:58.241068 23890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:58.236509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.131:46729 (local address 127.13.157.190:39553)
0925 02:43:58.236678 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:58.236682 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:58.237876 (+ 1194us) server_negotiation.cc:407] Connection header received
0925 02:43:58.239318 (+ 1442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:58.239322 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:58.239371 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:58.239458 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:58.240617 (+ 1159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:58.240620 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:58.240622 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:43:58.240634 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:58.240655 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:43:58.240663 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:58.240748 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:58.240875 (+ 127us) server_negotiation.cc:299] Negotiation successful
0925 02:43:58.240924 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260925 02:43:58.241436 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 24176
I20260925 02:43:58.241546 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790304178325258-13942-0/minicluster-data/ts-2/wal/instance
I20260925 02:43:58.241873 13942 external_mini_cluster.cc:1428] Setting key dadb7541743e40402113c8ebe7378ed5
I20260925 02:43:58.242192 24307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:58.236949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:39553 (local address 127.13.157.131:46729)
0925 02:43:58.237698 (+ 749us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:58.237751 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:58.239104 (+ 1353us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:58.239654 (+ 550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:58.239661 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:58.239757 (+ 96us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:58.240323 (+ 566us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 02:43:58.240325 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:58.240369 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 02:43:58.240441 (+ 72us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:43:58.240738 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:43:58.240746 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:43:58.240752 (+ 6us) client_negotiation.cc:769] Sending connection context
0925 02:43:58.240918 (+ 166us) client_negotiation.cc:240] Negotiation successful
0925 02:43:58.241313 (+ 395us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":109,"threads_started":1}
I20260925 02:43:58.243356 24305 heartbeater.cc:347] Connected to a master server at 127.13.157.190:39553
I20260925 02:43:58.243584 24305 heartbeater.cc:464] Registering TS with master...
I20260925 02:43:58.244023 24305 heartbeater.cc:511] Master 127.13.157.190:39553 requested a full tablet report, sending...
I20260925 02:43:58.245041 23851 ts_manager.cc:194] Registered new tserver with Master: 98c79c2e6d74468ca4e900f1dee1389d (127.13.157.131:40291)
I20260925 02:43:58.256366 13942 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260925 02:43:58.259409 23890 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:43:58.258382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59054 (local address 127.13.157.190:39553)
0925 02:43:58.258528 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:58.258532 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:58.258720 (+ 188us) server_negotiation.cc:407] Connection header received
0925 02:43:58.258800 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:58.258802 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:58.258840 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:58.258994 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:58.259264 (+ 270us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59054: BlockingRecv error: recv got EOF from 127.0.0.1:59054 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
I20260925 02:43:58.259521 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 23911
I20260925 02:43:58.264833 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 24044
I20260925 02:43:58.270183 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 24176
I20260925 02:43:58.274837 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 23819
2026-09-25T02:43:58Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (846 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790304178325258-13942-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 02:43:58 dist-test-slave-kznw krb5kdc[24315](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 02:43:58 dist-test-slave-kznw krb5kdc[24315](info): set up 2 sockets
Sep 25 02:43:58 dist-test-slave-kznw krb5kdc[24315](info): commencing operation
krb5kdc: starting...
W20260925 02:43:59.251472 13942 mini_kdc.cc:121] Time spent starting KDC: real 0.957s user 0.006s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790304178325258-13942-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790304178325258-13942-0/krb5kdc/test-user.keytab.
Sep 25 02:43:59 dist-test-slave-kznw krb5kdc[24315](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304239, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T02:43:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T02:43:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.157.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.157.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.157.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab.
Entry for principal kudu/127.13.157.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab.
I20260925 02:43:59.372044 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790304178325258-13942-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790304178325258-13942-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790304178325258-13942-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.157.190:38257
--webserver_interface=127.13.157.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab
--principal=kudu/127.13.157.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:46437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.157.190:38257
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:43:59.482992 24328 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:43:59.483273 24328 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:43:59.483330 24328 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:43:59.487306 24328 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:43:59.487360 24328 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:43:59.487385 24328 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:43:59.487407 24328 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:43:59.487432 24328 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:43:59.492610 24328 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:46437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790304178325258-13942-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790304178325258-13942-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.157.190:38257
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab
--principal=kudu/127.13.157.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.157.190:38257
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.157.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24328
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790304178325258-13942-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:43:59.493609 24328 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:43:59.494589 24328 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 02:43:59.503300 24328 server_base.cc:1034] running on GCE node
W20260925 02:43:59.503249 24334 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:59.503325 24333 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:59.503244 24336 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:43:59.504055 24328 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:43:59.505182 24328 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:43:59.506347 24328 hybrid_clock.cc:648] HybridClock initialized: now 1790304239506319 us; error 40 us; skew 500 ppm
Sep 25 02:43:59 dist-test-slave-kznw krb5kdc[24315](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304239, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:43:59.509217 24328 init.cc:377] Logged in from keytab as kudu/127.13.157.190@KRBTEST.COM (short username kudu)
I20260925 02:43:59.510124 24328 webserver.cc:460] Webserver started at http://127.13.157.190:37711/ using document root <none> and password file <none>
I20260925 02:43:59.510665 24328 fs_manager.cc:359] Metadata directory not provided
I20260925 02:43:59.510735 24328 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:43:59.510972 24328 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:43:59.513029 24328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790304178325258-13942-0/minicluster-data/master-0/data/instance:
uuid: "3556c13fa7cb40ad877870afc594123a"
format_stamp: "Formatted at 2026-09-25 02:43:59 on dist-test-slave-kznw"
server_key: "ff31c9a21d51156b660b62b39a31ee89"
server_key_iv: "157e1dde180f93812696a1cd85aba38c"
server_key_version: "encryptionkey@0"
I20260925 02:43:59.513604 24328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790304178325258-13942-0/minicluster-data/master-0/wal/instance:
uuid: "3556c13fa7cb40ad877870afc594123a"
format_stamp: "Formatted at 2026-09-25 02:43:59 on dist-test-slave-kznw"
server_key: "ff31c9a21d51156b660b62b39a31ee89"
server_key_iv: "157e1dde180f93812696a1cd85aba38c"
server_key_version: "encryptionkey@0"
I20260925 02:43:59.517271 24328 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260925 02:43:59.519714 24343 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:59.520867 24328 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260925 02:43:59.520977 24328 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790304178325258-13942-0/minicluster-data/master-0/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790304178325258-13942-0/minicluster-data/master-0/wal
uuid: "3556c13fa7cb40ad877870afc594123a"
format_stamp: "Formatted at 2026-09-25 02:43:59 on dist-test-slave-kznw"
server_key: "ff31c9a21d51156b660b62b39a31ee89"
server_key_iv: "157e1dde180f93812696a1cd85aba38c"
server_key_version: "encryptionkey@0"
I20260925 02:43:59.521067 24328 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790304178325258-13942-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790304178325258-13942-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790304178325258-13942-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:43:59.545943 24328 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:43:59.549120 24328 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:43:59.549330 24328 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:43:59.556438 24328 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.190:38257
I20260925 02:43:59.556720 24396 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.190:38257 every 8 connection(s)
I20260925 02:43:59.557444 24328 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
I20260925 02:43:59.558241 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 24328
I20260925 02:43:59.558369 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790304178325258-13942-0/minicluster-data/master-0/wal/instance
I20260925 02:43:59.558719 13942 external_mini_cluster.cc:1428] Setting key d51be388377b3f414c214899b01bc4a3
I20260925 02:43:59.560958 24397 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:43:59.566509 24397 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3556c13fa7cb40ad877870afc594123a: Bootstrap starting.
Sep 25 02:43:59 dist-test-slave-kznw krb5kdc[24315](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304239, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.157.190@KRBTEST.COM
I20260925 02:43:59.568857 24397 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3556c13fa7cb40ad877870afc594123a: Neither blocks nor log segments found. Creating new log.
I20260925 02:43:59.569581 24397 log.cc:824] T 00000000000000000000000000000000 P 3556c13fa7cb40ad877870afc594123a: Log is configured to *not* fsync() on all Append() calls
I20260925 02:43:59.571671 24397 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3556c13fa7cb40ad877870afc594123a: No bootstrap required, opened a new log
I20260925 02:43:59.572551 24400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:59.560287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38146 (local address 127.13.157.190:38257)
0925 02:43:59.560788 (+ 501us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:59.560804 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:59.560841 (+ 37us) server_negotiation.cc:407] Connection header received
0925 02:43:59.561522 (+ 681us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:59.561543 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:59.561883 (+ 340us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:59.562230 (+ 347us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:59.563065 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:59.564071 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:59.564922 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:59.565392 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:59.565536 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:59.567938 (+ 2402us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:59.567944 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:59.567956 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:43:59.567984 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:59.569554 (+ 1570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:59.569976 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:59.569981 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:59.569985 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:59.570109 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:59.570421 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:59.570424 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:59.570425 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:59.570858 (+ 433us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:59.571079 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:59.571420 (+ 341us) server_negotiation.cc:299] Negotiation successful
0925 02:43:59.571741 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":150,"threads_started":1}
I20260925 02:43:59.575201 24397 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3556c13fa7cb40ad877870afc594123a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3556c13fa7cb40ad877870afc594123a" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 38257 } }
I20260925 02:43:59.575420 24397 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3556c13fa7cb40ad877870afc594123a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:43:59.575467 24397 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3556c13fa7cb40ad877870afc594123a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3556c13fa7cb40ad877870afc594123a, State: Initialized, Role: FOLLOWER
I20260925 02:43:59.575886 24397 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3556c13fa7cb40ad877870afc594123a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3556c13fa7cb40ad877870afc594123a" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 38257 } }
I20260925 02:43:59.576014 24397 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3556c13fa7cb40ad877870afc594123a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 02:43:59.576077 24397 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3556c13fa7cb40ad877870afc594123a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 02:43:59.576174 24397 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3556c13fa7cb40ad877870afc594123a [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:43:59.577136 24397 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3556c13fa7cb40ad877870afc594123a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3556c13fa7cb40ad877870afc594123a" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 38257 } }
I20260925 02:43:59.577497 24397 leader_election.cc:302] T 00000000000000000000000000000000 P 3556c13fa7cb40ad877870afc594123a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3556c13fa7cb40ad877870afc594123a; no voters:
I20260925 02:43:59.577883 24397 leader_election.cc:288] T 00000000000000000000000000000000 P 3556c13fa7cb40ad877870afc594123a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 02:43:59.578049 24402 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3556c13fa7cb40ad877870afc594123a [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:43:59.578265 24402 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3556c13fa7cb40ad877870afc594123a [term 1 LEADER]: Becoming Leader. State: Replica: 3556c13fa7cb40ad877870afc594123a, State: Running, Role: LEADER
I20260925 02:43:59.578574 24402 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3556c13fa7cb40ad877870afc594123a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3556c13fa7cb40ad877870afc594123a" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 38257 } }
I20260925 02:43:59.579092 24397 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3556c13fa7cb40ad877870afc594123a [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:43:59.580143 24403 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3556c13fa7cb40ad877870afc594123a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3556c13fa7cb40ad877870afc594123a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3556c13fa7cb40ad877870afc594123a" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 38257 } } }
I20260925 02:43:59.580273 24403 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3556c13fa7cb40ad877870afc594123a [sys.catalog]: This master's current role is: LEADER
I20260925 02:43:59.580648 24411 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:43:59.580616 24404 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3556c13fa7cb40ad877870afc594123a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3556c13fa7cb40ad877870afc594123a. Latest consensus state: current_term: 1 leader_uuid: "3556c13fa7cb40ad877870afc594123a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3556c13fa7cb40ad877870afc594123a" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 38257 } } }
I20260925 02:43:59.581280 24404 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3556c13fa7cb40ad877870afc594123a [sys.catalog]: This master's current role is: LEADER
I20260925 02:43:59.583320 24411 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:43:59.588881 24411 catalog_manager.cc:1347] Generated new cluster ID: 82357aa9ab3243edb8d85a8d9a6e956e
I20260925 02:43:59.588960 24411 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 02:43:59.611536 24411 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 02:43:59.612828 24411 catalog_manager.cc:1504] Loading token signing keys...
I20260925 02:43:59.622551 24411 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3556c13fa7cb40ad877870afc594123a: Generated new TSK 0
I20260925 02:43:59.623447 24411 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.157.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.157.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.157.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.129.keytab.
Entry for principal kudu/127.13.157.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.129.keytab.
I20260925 02:43:59.655412 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790304178325258-13942-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790304178325258-13942-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790304178325258-13942-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790304178325258-13942-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.157.129:0
--local_ip_for_outbound_sockets=127.13.157.129
--webserver_interface=127.13.157.129
--webserver_port=0
--tserver_master_addrs=127.13.157.190:38257
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.129.keytab
--principal=kudu/127.13.157.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:46437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:43:59.767342 24423 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:43:59.767637 24423 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:43:59.767683 24423 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:43:59.771787 24423 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:43:59.771854 24423 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:43:59.771919 24423 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.157.129
I20260925 02:43:59.777519 24423 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:46437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790304178325258-13942-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790304178325258-13942-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.129.keytab
--principal=kudu/127.13.157.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.157.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790304178325258-13942-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.157.129
--webserver_port=0
--tserver_master_addrs=127.13.157.190:38257
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24423
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.157.129
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790304178325258-13942-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:43:59.778642 24423 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:43:59.779814 24423 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:43:59.790110 24431 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:59.790104 24429 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:43:59.790104 24428 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:43:59.790167 24423 server_base.cc:1034] running on GCE node
I20260925 02:43:59.790760 24423 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:43:59.791474 24423 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:43:59.792629 24423 hybrid_clock.cc:648] HybridClock initialized: now 1790304239792602 us; error 45 us; skew 500 ppm
Sep 25 02:43:59 dist-test-slave-kznw krb5kdc[24315](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304239, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:43:59.795440 24423 init.cc:377] Logged in from keytab as kudu/127.13.157.129@KRBTEST.COM (short username kudu)
I20260925 02:43:59.796319 24423 webserver.cc:460] Webserver started at http://127.13.157.129:44539/ using document root <none> and password file <none>
I20260925 02:43:59.796854 24423 fs_manager.cc:359] Metadata directory not provided
I20260925 02:43:59.796912 24423 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:43:59.797107 24423 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:43:59.799196 24423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790304178325258-13942-0/minicluster-data/ts-0/data/instance:
uuid: "1b6640624dd9445492ad7185990c1665"
format_stamp: "Formatted at 2026-09-25 02:43:59 on dist-test-slave-kznw"
server_key: "2115212fce5343e47ace056d2518e463"
server_key_iv: "e7458e336362bb78d68687ff2c49b9a8"
server_key_version: "encryptionkey@0"
I20260925 02:43:59.799750 24423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790304178325258-13942-0/minicluster-data/ts-0/wal/instance:
uuid: "1b6640624dd9445492ad7185990c1665"
format_stamp: "Formatted at 2026-09-25 02:43:59 on dist-test-slave-kznw"
server_key: "2115212fce5343e47ace056d2518e463"
server_key_iv: "e7458e336362bb78d68687ff2c49b9a8"
server_key_version: "encryptionkey@0"
I20260925 02:43:59.803364 24423 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260925 02:43:59.805763 24438 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:59.807020 24423 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260925 02:43:59.807123 24423 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790304178325258-13942-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790304178325258-13942-0/minicluster-data/ts-0/wal
uuid: "1b6640624dd9445492ad7185990c1665"
format_stamp: "Formatted at 2026-09-25 02:43:59 on dist-test-slave-kznw"
server_key: "2115212fce5343e47ace056d2518e463"
server_key_iv: "e7458e336362bb78d68687ff2c49b9a8"
server_key_version: "encryptionkey@0"
I20260925 02:43:59.807212 24423 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790304178325258-13942-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790304178325258-13942-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790304178325258-13942-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:43:59.842955 24423 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:43:59.846076 24423 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:43:59.846262 24423 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:43:59.846943 24423 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:43:59.847805 24423 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:43:59.847851 24423 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:59.847901 24423 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:43:59.847930 24423 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:43:59.856739 24423 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.129:34935
I20260925 02:43:59.856827 24551 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.129:34935 every 8 connection(s)
I20260925 02:43:59.857628 24423 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790304178325258-13942-0/minicluster-data/ts-0/data/info.pb
I20260925 02:43:59.862308 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 24423
I20260925 02:43:59.862437 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790304178325258-13942-0/minicluster-data/ts-0/wal/instance
I20260925 02:43:59.862785 13942 external_mini_cluster.cc:1428] Setting key 0b3f0b05e47969ce50e42f470f32ce49
Sep 25 02:43:59 dist-test-slave-kznw krb5kdc[24315](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304239, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.129@KRBTEST.COM for kudu/127.13.157.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.157.130@KRBTEST.COM; defaulting to no policy
I20260925 02:43:59.872529 24400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:59.859439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.129:50005 (local address 127.13.157.190:38257)
0925 02:43:59.859621 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0925 02:43:59.859626 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:43:59.860448 (+ 822us) server_negotiation.cc:407] Connection header received
0925 02:43:59.861796 (+ 1348us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:43:59.861799 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:43:59.861848 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:43:59.861928 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:43:59.863507 (+ 1579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:59.864279 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:59.865844 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:43:59.866098 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:43:59.866194 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:59.868889 (+ 2695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:43:59.868892 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:43:59.868894 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:43:59.868922 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:43:59.870465 (+ 1543us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:59.871000 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:59.871004 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:59.871006 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:59.871087 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:43:59.871410 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:43:59.871414 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:43:59.871415 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:43:59.871594 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:43:59.871678 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0925 02:43:59.872219 (+ 541us) server_negotiation.cc:299] Negotiation successful
0925 02:43:59.872352 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260925 02:43:59.873407 24554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:43:59.859827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:38257 (local address 127.13.157.129:50005)
0925 02:43:59.860298 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0925 02:43:59.860341 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0925 02:43:59.861594 (+ 1253us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:43:59.862163 (+ 569us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:43:59.862173 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:43:59.862695 (+ 522us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:43:59.863339 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:59.863359 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:59.864426 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:59.864430 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:59.865707 (+ 1277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:43:59.865718 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:43:59.866248 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:43:59.866252 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:43:59.866415 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:43:59.867052 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:43:59.867067 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:43:59.868689 (+ 1622us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:43:59.870587 (+ 1898us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:59.870594 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:59.870610 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:59.870869 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:59.871171 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:43:59.871174 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:43:59.871176 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:43:59.871315 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:43:59.871692 (+ 377us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:43:59.871699 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:43:59.871968 (+ 269us) client_negotiation.cc:769] Sending connection context
0925 02:43:59.872148 (+ 180us) client_negotiation.cc:240] Negotiation successful
0925 02:43:59.872492 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":113,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.157.130@KRBTEST.COM" created.
I20260925 02:43:59.874861 24552 heartbeater.cc:347] Connected to a master server at 127.13.157.190:38257
I20260925 02:43:59.875195 24552 heartbeater.cc:464] Registering TS with master...
I20260925 02:43:59.875775 24552 heartbeater.cc:511] Master 127.13.157.190:38257 requested a full tablet report, sending...
I20260925 02:43:59.877529 24360 ts_manager.cc:194] Registered new tserver with Master: 1b6640624dd9445492ad7185990c1665 (127.13.157.129:34935)
I20260925 02:43:59.878793 24360 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.157.129@KRBTEST.COM'} at 127.13.157.129:50005
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.157.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.130.keytab.
Entry for principal kudu/127.13.157.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.130.keytab.
I20260925 02:43:59.890285 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790304178325258-13942-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790304178325258-13942-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790304178325258-13942-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790304178325258-13942-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.157.130:0
--local_ip_for_outbound_sockets=127.13.157.130
--webserver_interface=127.13.157.130
--webserver_port=0
--tserver_master_addrs=127.13.157.190:38257
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.130.keytab
--principal=kudu/127.13.157.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:46437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:44:00.000034 24557 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:44:00.000278 24557 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:44:00.000317 24557 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:44:00.004185 24557 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:44:00.004236 24557 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:44:00.004290 24557 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.157.130
I20260925 02:44:00.010480 24557 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:46437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790304178325258-13942-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790304178325258-13942-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.130.keytab
--principal=kudu/127.13.157.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.157.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790304178325258-13942-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.157.130
--webserver_port=0
--tserver_master_addrs=127.13.157.190:38257
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24557
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.157.130
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790304178325258-13942-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:44:00.011739 24557 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:44:00.012985 24557 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:44:00.023051 24562 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:44:00.023074 24565 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:44:00.023295 24557 server_base.cc:1034] running on GCE node
W20260925 02:44:00.023428 24563 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:44:00.023880 24557 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:44:00.024636 24557 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:44:00.025833 24557 hybrid_clock.cc:648] HybridClock initialized: now 1790304240025772 us; error 71 us; skew 500 ppm
Sep 25 02:44:00 dist-test-slave-kznw krb5kdc[24315](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304240, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:44:00.028784 24557 init.cc:377] Logged in from keytab as kudu/127.13.157.130@KRBTEST.COM (short username kudu)
I20260925 02:44:00.029767 24557 webserver.cc:460] Webserver started at http://127.13.157.130:42405/ using document root <none> and password file <none>
I20260925 02:44:00.030344 24557 fs_manager.cc:359] Metadata directory not provided
I20260925 02:44:00.030413 24557 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:44:00.030637 24557 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:44:00.032840 24557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790304178325258-13942-0/minicluster-data/ts-1/data/instance:
uuid: "7ef0d92a5e4c4b8482e1c52e9ed0b620"
format_stamp: "Formatted at 2026-09-25 02:44:00 on dist-test-slave-kznw"
server_key: "f4c6768932f2b3452334824d5fbf0e4c"
server_key_iv: "54ffb5ced86cf13dbda16a0afe1f76bb"
server_key_version: "encryptionkey@0"
I20260925 02:44:00.033418 24557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790304178325258-13942-0/minicluster-data/ts-1/wal/instance:
uuid: "7ef0d92a5e4c4b8482e1c52e9ed0b620"
format_stamp: "Formatted at 2026-09-25 02:44:00 on dist-test-slave-kznw"
server_key: "f4c6768932f2b3452334824d5fbf0e4c"
server_key_iv: "54ffb5ced86cf13dbda16a0afe1f76bb"
server_key_version: "encryptionkey@0"
I20260925 02:44:00.037587 24557 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260925 02:44:00.040266 24572 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:44:00.041553 24557 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260925 02:44:00.041668 24557 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790304178325258-13942-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790304178325258-13942-0/minicluster-data/ts-1/wal
uuid: "7ef0d92a5e4c4b8482e1c52e9ed0b620"
format_stamp: "Formatted at 2026-09-25 02:44:00 on dist-test-slave-kznw"
server_key: "f4c6768932f2b3452334824d5fbf0e4c"
server_key_iv: "54ffb5ced86cf13dbda16a0afe1f76bb"
server_key_version: "encryptionkey@0"
I20260925 02:44:00.041775 24557 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790304178325258-13942-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790304178325258-13942-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790304178325258-13942-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:44:00.052983 24557 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:44:00.056114 24557 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:44:00.056301 24557 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:44:00.056921 24557 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:44:00.057749 24557 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:44:00.057796 24557 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:44:00.057852 24557 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:44:00.057881 24557 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:44:00.066123 24557 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.130:33565
I20260925 02:44:00.066166 24687 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.130:33565 every 8 connection(s)
I20260925 02:44:00.067047 24557 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790304178325258-13942-0/minicluster-data/ts-1/data/info.pb
I20260925 02:44:00.076642 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 24557
I20260925 02:44:00.076745 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790304178325258-13942-0/minicluster-data/ts-1/wal/instance
Sep 25 02:44:00 dist-test-slave-kznw krb5kdc[24315](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304240, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.130@KRBTEST.COM for kudu/127.13.157.190@KRBTEST.COM
I20260925 02:44:00.077033 13942 external_mini_cluster.cc:1428] Setting key deec5ca318d8996f091ea86775952466
I20260925 02:44:00.081538 24400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:44:00.068902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.130:45893 (local address 127.13.157.190:38257)
0925 02:44:00.069087 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0925 02:44:00.069092 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:44:00.069893 (+ 801us) server_negotiation.cc:407] Connection header received
0925 02:44:00.071301 (+ 1408us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:44:00.071304 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:44:00.071353 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:44:00.071425 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:44:00.072682 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:44:00.073266 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:44:00.074465 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:44:00.074704 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:44:00.074796 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:44:00.077391 (+ 2595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:44:00.077395 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:44:00.077398 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:44:00.077426 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:44:00.079137 (+ 1711us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:44:00.079676 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:44:00.079680 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:44:00.079682 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:44:00.079782 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:44:00.080302 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:44:00.080304 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:44:00.080306 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:44:00.080502 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:44:00.080585 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0925 02:44:00.081246 (+ 661us) server_negotiation.cc:299] Negotiation successful
0925 02:44:00.081367 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260925 02:44:00.082674 24690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:44:00.069289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:38257 (local address 127.13.157.130:45893)
0925 02:44:00.069749 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0925 02:44:00.069791 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0925 02:44:00.071103 (+ 1312us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:44:00.071583 (+ 480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:44:00.071590 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:44:00.072013 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:44:00.072513 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:44:00.072528 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:44:00.073384 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:44:00.073387 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:44:00.074284 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:44:00.074294 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:44:00.074861 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:44:00.074864 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:44:00.075079 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:44:00.075654 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:44:00.075671 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:44:00.077204 (+ 1533us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:44:00.079259 (+ 2055us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:44:00.079266 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:44:00.079284 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:44:00.079567 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:44:00.079896 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:44:00.079900 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:44:00.079902 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:44:00.080215 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:44:00.080602 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:44:00.080610 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:44:00.080966 (+ 356us) client_negotiation.cc:769] Sending connection context
0925 02:44:00.081192 (+ 226us) client_negotiation.cc:240] Negotiation successful
0925 02:44:00.081571 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":114,"threads_started":1}
I20260925 02:44:00.083961 24688 heartbeater.cc:347] Connected to a master server at 127.13.157.190:38257
I20260925 02:44:00.084259 24688 heartbeater.cc:464] Registering TS with master...
I20260925 02:44:00.084767 24688 heartbeater.cc:511] Master 127.13.157.190:38257 requested a full tablet report, sending...
I20260925 02:44:00.085862 24360 ts_manager.cc:194] Registered new tserver with Master: 7ef0d92a5e4c4b8482e1c52e9ed0b620 (127.13.157.130:33565)
I20260925 02:44:00.086452 24360 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.157.130@KRBTEST.COM'} at 127.13.157.130:45893
WARNING: no policy specified for kudu/127.13.157.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.157.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.157.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.131.keytab.
Entry for principal kudu/127.13.157.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.131.keytab.
I20260925 02:44:00.106081 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790304178325258-13942-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790304178325258-13942-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790304178325258-13942-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790304178325258-13942-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.157.131:0
--local_ip_for_outbound_sockets=127.13.157.131
--webserver_interface=127.13.157.131
--webserver_port=0
--tserver_master_addrs=127.13.157.190:38257
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.131.keytab
--principal=kudu/127.13.157.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:46437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:44:00.216130 24693 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:44:00.216406 24693 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:44:00.216468 24693 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:44:00.220531 24693 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:44:00.220592 24693 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:44:00.220674 24693 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.157.131
I20260925 02:44:00.226051 24693 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:46437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790304178325258-13942-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790304178325258-13942-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.131.keytab
--principal=kudu/127.13.157.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.157.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790304178325258-13942-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.157.131
--webserver_port=0
--tserver_master_addrs=127.13.157.190:38257
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24693
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.157.131
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790304178325258-13942-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:44:00.227181 24693 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:44:00.228229 24693 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:44:00.238325 24701 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:44:00.238320 24699 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:44:00.238452 24693 server_base.cc:1034] running on GCE node
W20260925 02:44:00.238315 24698 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:44:00.239123 24693 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:44:00.239797 24693 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:44:00.240980 24693 hybrid_clock.cc:648] HybridClock initialized: now 1790304240240949 us; error 48 us; skew 500 ppm
Sep 25 02:44:00 dist-test-slave-kznw krb5kdc[24315](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304240, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:44:00.243818 24693 init.cc:377] Logged in from keytab as kudu/127.13.157.131@KRBTEST.COM (short username kudu)
I20260925 02:44:00.244751 24693 webserver.cc:460] Webserver started at http://127.13.157.131:46127/ using document root <none> and password file <none>
I20260925 02:44:00.245352 24693 fs_manager.cc:359] Metadata directory not provided
I20260925 02:44:00.245425 24693 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:44:00.245631 24693 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:44:00.248077 24693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790304178325258-13942-0/minicluster-data/ts-2/data/instance:
uuid: "1b69d6babf064b30a674b864aee85903"
format_stamp: "Formatted at 2026-09-25 02:44:00 on dist-test-slave-kznw"
server_key: "2261ab322d72b8a25e583491e1942b1c"
server_key_iv: "1b360c9efc915edb8d63b37a3269b884"
server_key_version: "encryptionkey@0"
I20260925 02:44:00.248660 24693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790304178325258-13942-0/minicluster-data/ts-2/wal/instance:
uuid: "1b69d6babf064b30a674b864aee85903"
format_stamp: "Formatted at 2026-09-25 02:44:00 on dist-test-slave-kznw"
server_key: "2261ab322d72b8a25e583491e1942b1c"
server_key_iv: "1b360c9efc915edb8d63b37a3269b884"
server_key_version: "encryptionkey@0"
I20260925 02:44:00.252642 24693 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260925 02:44:00.255343 24708 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:44:00.256783 24693 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260925 02:44:00.256923 24693 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790304178325258-13942-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790304178325258-13942-0/minicluster-data/ts-2/wal
uuid: "1b69d6babf064b30a674b864aee85903"
format_stamp: "Formatted at 2026-09-25 02:44:00 on dist-test-slave-kznw"
server_key: "2261ab322d72b8a25e583491e1942b1c"
server_key_iv: "1b360c9efc915edb8d63b37a3269b884"
server_key_version: "encryptionkey@0"
I20260925 02:44:00.257036 24693 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790304178325258-13942-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790304178325258-13942-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790304178325258-13942-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:44:00.276881 24693 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:44:00.280311 24693 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:44:00.280493 24693 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:44:00.281106 24693 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:44:00.281917 24693 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:44:00.281972 24693 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:44:00.282028 24693 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:44:00.282052 24693 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:44:00.290792 24693 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.131:33603
I20260925 02:44:00.290820 24823 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.131:33603 every 8 connection(s)
I20260925 02:44:00.291746 24693 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790304178325258-13942-0/minicluster-data/ts-2/data/info.pb
I20260925 02:44:00.292567 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 24693
I20260925 02:44:00.292680 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790304178325258-13942-0/minicluster-data/ts-2/wal/instance
I20260925 02:44:00.292972 13942 external_mini_cluster.cc:1428] Setting key 084b81180758928874721ebbcbbe0136
Sep 25 02:44:00 dist-test-slave-kznw krb5kdc[24315](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304240, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.131@KRBTEST.COM for kudu/127.13.157.190@KRBTEST.COM
I20260925 02:44:00.305476 24400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:44:00.293659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.131:41383 (local address 127.13.157.190:38257)
0925 02:44:00.293788 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0925 02:44:00.293792 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:44:00.294820 (+ 1028us) server_negotiation.cc:407] Connection header received
0925 02:44:00.296251 (+ 1431us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:44:00.296254 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:44:00.296300 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:44:00.296362 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:44:00.297495 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:44:00.298091 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:44:00.299383 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:44:00.299625 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:44:00.299714 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:44:00.302202 (+ 2488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:44:00.302205 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:44:00.302207 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:44:00.302232 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:44:00.303567 (+ 1335us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:44:00.304043 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:44:00.304045 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:44:00.304046 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:44:00.304120 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:44:00.304481 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:44:00.304484 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:44:00.304485 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:44:00.304662 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:44:00.304738 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0925 02:44:00.305191 (+ 453us) server_negotiation.cc:299] Negotiation successful
0925 02:44:00.305320 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260925 02:44:00.306396 24826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:44:00.294132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:38257 (local address 127.13.157.131:41383)
0925 02:44:00.294662 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0925 02:44:00.294716 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0925 02:44:00.296059 (+ 1343us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:44:00.296505 (+ 446us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:44:00.296511 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:44:00.296857 (+ 346us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:44:00.297336 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:44:00.297349 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:44:00.298237 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:44:00.298241 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:44:00.299245 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:44:00.299256 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:44:00.299789 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:44:00.299792 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:44:00.299959 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:44:00.300565 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:44:00.300582 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:44:00.302018 (+ 1436us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:44:00.303684 (+ 1666us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:44:00.303689 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:44:00.303700 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:44:00.303952 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:44:00.304216 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:44:00.304219 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:44:00.304220 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:44:00.304372 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:44:00.304745 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:44:00.304749 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:44:00.304982 (+ 233us) client_negotiation.cc:769] Sending connection context
0925 02:44:00.305139 (+ 157us) client_negotiation.cc:240] Negotiation successful
0925 02:44:00.305423 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":120,"threads_started":1}
I20260925 02:44:00.307554 24824 heartbeater.cc:347] Connected to a master server at 127.13.157.190:38257
I20260925 02:44:00.307794 24824 heartbeater.cc:464] Registering TS with master...
I20260925 02:44:00.308285 24824 heartbeater.cc:511] Master 127.13.157.190:38257 requested a full tablet report, sending...
I20260925 02:44:00.309384 24360 ts_manager.cc:194] Registered new tserver with Master: 1b69d6babf064b30a674b864aee85903 (127.13.157.131:33603)
I20260925 02:44:00.309950 24360 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.157.131@KRBTEST.COM'} at 127.13.157.131:41383
I20260925 02:44:00.319164 13942 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 02:44:00.328748 24400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:44:00.321247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38148 (local address 127.13.157.190:38257)
0925 02:44:00.321384 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0925 02:44:00.321388 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:44:00.321513 (+ 125us) server_negotiation.cc:407] Connection header received
0925 02:44:00.321591 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:44:00.321594 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:44:00.321646 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:44:00.321734 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:44:00.322524 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:44:00.323182 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:44:00.324123 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:44:00.324355 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:44:00.324440 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:44:00.325780 (+ 1340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:44:00.325783 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:44:00.325785 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:44:00.325811 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:44:00.327143 (+ 1332us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:44:00.327542 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:44:00.327546 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:44:00.327547 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:44:00.327632 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:44:00.327960 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:44:00.327963 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:44:00.327964 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:44:00.328185 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:44:00.328292 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0925 02:44:00.328528 (+ 236us) server_negotiation.cc:299] Negotiation successful
0925 02:44:00.328628 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260925 02:44:00.331758 24360 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38148:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 02:44:00.334817 24360 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 02:44:00.349707 24835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:44:00.343462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.129:34935 (local address 127.0.0.1:57114)
0925 02:44:00.344007 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0925 02:44:00.344117 (+ 110us) client_negotiation.cc:174] Beginning negotiation
0925 02:44:00.344257 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:44:00.345054 (+ 797us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:44:00.345060 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:44:00.345084 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:44:00.345357 (+ 273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:44:00.345364 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:44:00.346895 (+ 1531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:44:00.346899 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:44:00.348264 (+ 1365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:44:00.348274 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:44:00.349242 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:44:00.349245 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:44:00.349335 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:44:00.349367 (+ 32us) client_negotiation.cc:769] Sending connection context
0925 02:44:00.349454 (+ 87us) client_negotiation.cc:240] Negotiation successful
0925 02:44:00.349548 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"mutex_wait_us":219,"thread_start_us":99,"threads_started":1}
I20260925 02:44:00.349756 24836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:44:00.343633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57114 (local address 127.13.157.129:34935)
0925 02:44:00.344183 (+ 550us) server_negotiation.cc:206] Beginning negotiation
0925 02:44:00.344202 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0925 02:44:00.344229 (+ 27us) server_negotiation.cc:407] Connection header received
0925 02:44:00.344471 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:44:00.344478 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:44:00.344715 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:44:00.344883 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:44:00.345480 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:44:00.346745 (+ 1265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:44:00.348381 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:44:00.349113 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:44:00.349245 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:44:00.349382 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0925 02:44:00.349488 (+ 106us) server_negotiation.cc:299] Negotiation successful
0925 02:44:00.349603 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":430,"thread_start_us":68,"threads_started":1}
I20260925 02:44:00.350961 24837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:44:00.343821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.131:33603 (local address 127.0.0.1:52222)
0925 02:44:00.344436 (+ 615us) negotiation.cc:107] Waiting for socket to connect
0925 02:44:00.344448 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 02:44:00.344520 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:44:00.345102 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:44:00.345105 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:44:00.345118 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:44:00.345249 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:44:00.345257 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:44:00.346890 (+ 1633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:44:00.346894 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:44:00.348568 (+ 1674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:44:00.348579 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:44:00.350524 (+ 1945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:44:00.350528 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:44:00.350628 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:44:00.350651 (+ 23us) client_negotiation.cc:769] Sending connection context
0925 02:44:00.350726 (+ 75us) client_negotiation.cc:240] Negotiation successful
0925 02:44:00.350794 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":556,"mutex_wait_us":213,"thread_start_us":84,"threads_started":1}
I20260925 02:44:00.351034 24839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:44:00.344282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52222 (local address 127.13.157.131:33603)
0925 02:44:00.344685 (+ 403us) server_negotiation.cc:206] Beginning negotiation
0925 02:44:00.344697 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 02:44:00.344719 (+ 22us) server_negotiation.cc:407] Connection header received
0925 02:44:00.344795 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:44:00.344800 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:44:00.344987 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:44:00.345140 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:44:00.345552 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:44:00.346737 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:44:00.349800 (+ 3063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:44:00.350399 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:44:00.350531 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:44:00.350662 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0925 02:44:00.350752 (+ 90us) server_negotiation.cc:299] Negotiation successful
0925 02:44:00.350851 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":84,"threads_started":1}
I20260925 02:44:00.352743 24486 tablet_service.cc:1467] Processing CreateTablet for tablet 3589a37edf7540afa56060a24ad7e381 (DEFAULT_TABLE table=test-table [id=c2148a15bac84c3eb8a457060a83b97b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:44:00.353240 24757 tablet_service.cc:1467] Processing CreateTablet for tablet 3589a37edf7540afa56060a24ad7e381 (DEFAULT_TABLE table=test-table [id=c2148a15bac84c3eb8a457060a83b97b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:44:00.353696 24486 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3589a37edf7540afa56060a24ad7e381. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:44:00.354067 24757 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3589a37edf7540afa56060a24ad7e381. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:44:00.354887 24838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:44:00.344207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.130:33565 (local address 127.0.0.1:34694)
0925 02:44:00.345184 (+ 977us) negotiation.cc:107] Waiting for socket to connect
0925 02:44:00.345197 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 02:44:00.345274 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:44:00.345782 (+ 508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:44:00.345784 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:44:00.345800 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:44:00.345892 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:44:00.345898 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:44:00.347022 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:44:00.347025 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:44:00.348568 (+ 1543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:44:00.348579 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:44:00.349630 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:44:00.349634 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:44:00.349710 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:44:00.349732 (+ 22us) client_negotiation.cc:769] Sending connection context
0925 02:44:00.354664 (+ 4932us) client_negotiation.cc:240] Negotiation successful
0925 02:44:00.354751 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":914,"mutex_wait_us":131,"thread_start_us":85,"threads_started":1}
I20260925 02:44:00.355384 24840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:44:00.344738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34694 (local address 127.13.157.130:33565)
0925 02:44:00.345393 (+ 655us) server_negotiation.cc:206] Beginning negotiation
0925 02:44:00.345404 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 02:44:00.345427 (+ 23us) server_negotiation.cc:407] Connection header received
0925 02:44:00.345501 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:44:00.345506 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:44:00.345665 (+ 159us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:44:00.345786 (+ 121us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:44:00.345996 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:44:00.346864 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:44:00.348696 (+ 1832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:44:00.349506 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:44:00.349621 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:44:00.349737 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0925 02:44:00.355109 (+ 5372us) server_negotiation.cc:299] Negotiation successful
0925 02:44:00.355234 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":525,"thread_start_us":100,"threads_started":1}
I20260925 02:44:00.358455 24621 tablet_service.cc:1467] Processing CreateTablet for tablet 3589a37edf7540afa56060a24ad7e381 (DEFAULT_TABLE table=test-table [id=c2148a15bac84c3eb8a457060a83b97b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:44:00.359634 24621 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3589a37edf7540afa56060a24ad7e381. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:44:00.360517 24841 tablet_bootstrap.cc:492] T 3589a37edf7540afa56060a24ad7e381 P 1b69d6babf064b30a674b864aee85903: Bootstrap starting.
I20260925 02:44:00.361974 24842 tablet_bootstrap.cc:492] T 3589a37edf7540afa56060a24ad7e381 P 1b6640624dd9445492ad7185990c1665: Bootstrap starting.
I20260925 02:44:00.362716 24841 tablet_bootstrap.cc:654] T 3589a37edf7540afa56060a24ad7e381 P 1b69d6babf064b30a674b864aee85903: Neither blocks nor log segments found. Creating new log.
I20260925 02:44:00.363827 24841 log.cc:824] T 3589a37edf7540afa56060a24ad7e381 P 1b69d6babf064b30a674b864aee85903: Log is configured to *not* fsync() on all Append() calls
I20260925 02:44:00.364190 24842 tablet_bootstrap.cc:654] T 3589a37edf7540afa56060a24ad7e381 P 1b6640624dd9445492ad7185990c1665: Neither blocks nor log segments found. Creating new log.
I20260925 02:44:00.365471 24842 log.cc:824] T 3589a37edf7540afa56060a24ad7e381 P 1b6640624dd9445492ad7185990c1665: Log is configured to *not* fsync() on all Append() calls
I20260925 02:44:00.366291 24841 tablet_bootstrap.cc:492] T 3589a37edf7540afa56060a24ad7e381 P 1b69d6babf064b30a674b864aee85903: No bootstrap required, opened a new log
I20260925 02:44:00.366482 24841 ts_tablet_manager.cc:1397] T 3589a37edf7540afa56060a24ad7e381 P 1b69d6babf064b30a674b864aee85903: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260925 02:44:00.367575 24845 tablet_bootstrap.cc:492] T 3589a37edf7540afa56060a24ad7e381 P 7ef0d92a5e4c4b8482e1c52e9ed0b620: Bootstrap starting.
I20260925 02:44:00.367622 24842 tablet_bootstrap.cc:492] T 3589a37edf7540afa56060a24ad7e381 P 1b6640624dd9445492ad7185990c1665: No bootstrap required, opened a new log
I20260925 02:44:00.367800 24842 ts_tablet_manager.cc:1397] T 3589a37edf7540afa56060a24ad7e381 P 1b6640624dd9445492ad7185990c1665: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.001s
I20260925 02:44:00.369560 24841 raft_consensus.cc:348] T 3589a37edf7540afa56060a24ad7e381 P 1b69d6babf064b30a674b864aee85903 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b6640624dd9445492ad7185990c1665" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 34935 } } peers { permanent_uuid: "7ef0d92a5e4c4b8482e1c52e9ed0b620" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33565 } } peers { permanent_uuid: "1b69d6babf064b30a674b864aee85903" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 33603 } }
I20260925 02:44:00.369779 24841 raft_consensus.cc:374] T 3589a37edf7540afa56060a24ad7e381 P 1b69d6babf064b30a674b864aee85903 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:44:00.369833 24841 raft_consensus.cc:729] T 3589a37edf7540afa56060a24ad7e381 P 1b69d6babf064b30a674b864aee85903 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b69d6babf064b30a674b864aee85903, State: Initialized, Role: FOLLOWER
I20260925 02:44:00.370265 24845 tablet_bootstrap.cc:654] T 3589a37edf7540afa56060a24ad7e381 P 7ef0d92a5e4c4b8482e1c52e9ed0b620: Neither blocks nor log segments found. Creating new log.
I20260925 02:44:00.370253 24841 consensus_queue.cc:260] T 3589a37edf7540afa56060a24ad7e381 P 1b69d6babf064b30a674b864aee85903 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b6640624dd9445492ad7185990c1665" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 34935 } } peers { permanent_uuid: "7ef0d92a5e4c4b8482e1c52e9ed0b620" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33565 } } peers { permanent_uuid: "1b69d6babf064b30a674b864aee85903" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 33603 } }
I20260925 02:44:00.370936 24824 heartbeater.cc:503] Master 127.13.157.190:38257 was elected leader, sending a full tablet report...
I20260925 02:44:00.371228 24841 ts_tablet_manager.cc:1428] T 3589a37edf7540afa56060a24ad7e381 P 1b69d6babf064b30a674b864aee85903: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
I20260925 02:44:00.371625 24842 raft_consensus.cc:348] T 3589a37edf7540afa56060a24ad7e381 P 1b6640624dd9445492ad7185990c1665 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b6640624dd9445492ad7185990c1665" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 34935 } } peers { permanent_uuid: "7ef0d92a5e4c4b8482e1c52e9ed0b620" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33565 } } peers { permanent_uuid: "1b69d6babf064b30a674b864aee85903" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 33603 } }
I20260925 02:44:00.371780 24845 log.cc:824] T 3589a37edf7540afa56060a24ad7e381 P 7ef0d92a5e4c4b8482e1c52e9ed0b620: Log is configured to *not* fsync() on all Append() calls
I20260925 02:44:00.371848 24842 raft_consensus.cc:374] T 3589a37edf7540afa56060a24ad7e381 P 1b6640624dd9445492ad7185990c1665 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:44:00.371896 24842 raft_consensus.cc:729] T 3589a37edf7540afa56060a24ad7e381 P 1b6640624dd9445492ad7185990c1665 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b6640624dd9445492ad7185990c1665, State: Initialized, Role: FOLLOWER
I20260925 02:44:00.373129 24842 consensus_queue.cc:260] T 3589a37edf7540afa56060a24ad7e381 P 1b6640624dd9445492ad7185990c1665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b6640624dd9445492ad7185990c1665" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 34935 } } peers { permanent_uuid: "7ef0d92a5e4c4b8482e1c52e9ed0b620" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33565 } } peers { permanent_uuid: "1b69d6babf064b30a674b864aee85903" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 33603 } }
I20260925 02:44:00.374063 24842 ts_tablet_manager.cc:1428] T 3589a37edf7540afa56060a24ad7e381 P 1b6640624dd9445492ad7185990c1665: Time spent starting tablet: real 0.006s user 0.003s sys 0.002s
I20260925 02:44:00.374419 24845 tablet_bootstrap.cc:492] T 3589a37edf7540afa56060a24ad7e381 P 7ef0d92a5e4c4b8482e1c52e9ed0b620: No bootstrap required, opened a new log
I20260925 02:44:00.374439 24552 heartbeater.cc:503] Master 127.13.157.190:38257 was elected leader, sending a full tablet report...
I20260925 02:44:00.374629 24845 ts_tablet_manager.cc:1397] T 3589a37edf7540afa56060a24ad7e381 P 7ef0d92a5e4c4b8482e1c52e9ed0b620: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260925 02:44:00.378222 24845 raft_consensus.cc:348] T 3589a37edf7540afa56060a24ad7e381 P 7ef0d92a5e4c4b8482e1c52e9ed0b620 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b6640624dd9445492ad7185990c1665" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 34935 } } peers { permanent_uuid: "7ef0d92a5e4c4b8482e1c52e9ed0b620" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33565 } } peers { permanent_uuid: "1b69d6babf064b30a674b864aee85903" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 33603 } }
I20260925 02:44:00.378453 24845 raft_consensus.cc:374] T 3589a37edf7540afa56060a24ad7e381 P 7ef0d92a5e4c4b8482e1c52e9ed0b620 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:44:00.378486 24845 raft_consensus.cc:729] T 3589a37edf7540afa56060a24ad7e381 P 7ef0d92a5e4c4b8482e1c52e9ed0b620 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ef0d92a5e4c4b8482e1c52e9ed0b620, State: Initialized, Role: FOLLOWER
I20260925 02:44:00.378945 24845 consensus_queue.cc:260] T 3589a37edf7540afa56060a24ad7e381 P 7ef0d92a5e4c4b8482e1c52e9ed0b620 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b6640624dd9445492ad7185990c1665" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 34935 } } peers { permanent_uuid: "7ef0d92a5e4c4b8482e1c52e9ed0b620" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33565 } } peers { permanent_uuid: "1b69d6babf064b30a674b864aee85903" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 33603 } }
I20260925 02:44:00.379621 24688 heartbeater.cc:503] Master 127.13.157.190:38257 was elected leader, sending a full tablet report...
I20260925 02:44:00.379946 24845 ts_tablet_manager.cc:1428] T 3589a37edf7540afa56060a24ad7e381 P 7ef0d92a5e4c4b8482e1c52e9ed0b620: Time spent starting tablet: real 0.005s user 0.007s sys 0.000s
I20260925 02:44:00.543502 24849 raft_consensus.cc:482] T 3589a37edf7540afa56060a24ad7e381 P 7ef0d92a5e4c4b8482e1c52e9ed0b620 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260925 02:44:00.543563 24825 tablet.cc:2369] T 3589a37edf7540afa56060a24ad7e381 P 1b69d6babf064b30a674b864aee85903: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 02:44:00.543748 24849 raft_consensus.cc:504] T 3589a37edf7540afa56060a24ad7e381 P 7ef0d92a5e4c4b8482e1c52e9ed0b620 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b6640624dd9445492ad7185990c1665" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 34935 } } peers { permanent_uuid: "7ef0d92a5e4c4b8482e1c52e9ed0b620" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33565 } } peers { permanent_uuid: "1b69d6babf064b30a674b864aee85903" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 33603 } }
I20260925 02:44:00.545137 24849 leader_election.cc:288] T 3589a37edf7540afa56060a24ad7e381 P 7ef0d92a5e4c4b8482e1c52e9ed0b620 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1b6640624dd9445492ad7185990c1665 (127.13.157.129:34935), 1b69d6babf064b30a674b864aee85903 (127.13.157.131:33603)
I20260925 02:44:00.550035 24839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:44:00.545506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.130:48757 (local address 127.13.157.131:33603)
0925 02:44:00.545643 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0925 02:44:00.545648 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:44:00.545666 (+ 18us) server_negotiation.cc:407] Connection header received
0925 02:44:00.545831 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:44:00.545834 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:44:00.545882 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:44:00.545953 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:44:00.546213 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:44:00.547052 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:44:00.548679 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:44:00.549312 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:44:00.549495 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:44:00.549550 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0925 02:44:00.549811 (+ 261us) server_negotiation.cc:299] Negotiation successful
0925 02:44:00.549894 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260925 02:44:00.550067 24836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:44:00.545527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.130:58189 (local address 127.13.157.129:34935)
0925 02:44:00.545646 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0925 02:44:00.545650 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:44:00.545956 (+ 306us) server_negotiation.cc:407] Connection header received
0925 02:44:00.546064 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:44:00.546068 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:44:00.546108 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:44:00.546162 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:44:00.546396 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:44:00.547198 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:44:00.548679 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:44:00.549364 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:44:00.549474 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:44:00.549536 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0925 02:44:00.549811 (+ 275us) server_negotiation.cc:299] Negotiation successful
0925 02:44:00.549894 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260925 02:44:00.550125 24690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:44:00.545437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.131:33603 (local address 127.13.157.130:48757)
0925 02:44:00.545576 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0925 02:44:00.545593 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 02:44:00.545703 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:44:00.545977 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:44:00.545980 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:44:00.545994 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:44:00.546115 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:44:00.546120 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:44:00.547196 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:44:00.547199 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:44:00.548519 (+ 1320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:44:00.548534 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:44:00.549470 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:44:00.549474 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:44:00.549637 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:44:00.549691 (+ 54us) client_negotiation.cc:769] Sending connection context
0925 02:44:00.549771 (+ 80us) client_negotiation.cc:240] Negotiation successful
0925 02:44:00.549914 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57,"mutex_wait_us":53}
I20260925 02:44:00.550132 24850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:44:00.545443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.129:34935 (local address 127.13.157.130:58189)
0925 02:44:00.545845 (+ 402us) negotiation.cc:107] Waiting for socket to connect
0925 02:44:00.545856 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 02:44:00.545970 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:44:00.546195 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:44:00.546198 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:44:00.546214 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:44:00.546300 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:44:00.546305 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:44:00.547308 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:44:00.547311 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:44:00.548533 (+ 1222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:44:00.548540 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:44:00.549477 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:44:00.549480 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:44:00.549640 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:44:00.549698 (+ 58us) client_negotiation.cc:769] Sending connection context
0925 02:44:00.549777 (+ 79us) client_negotiation.cc:240] Negotiation successful
0925 02:44:00.549914 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":344,"mutex_wait_us":286,"thread_start_us":86,"threads_started":1}
I20260925 02:44:00.550858 24777 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3589a37edf7540afa56060a24ad7e381" candidate_uuid: "7ef0d92a5e4c4b8482e1c52e9ed0b620" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b69d6babf064b30a674b864aee85903" is_pre_election: true
I20260925 02:44:00.550889 24506 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3589a37edf7540afa56060a24ad7e381" candidate_uuid: "7ef0d92a5e4c4b8482e1c52e9ed0b620" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b6640624dd9445492ad7185990c1665" is_pre_election: true
I20260925 02:44:00.551216 24506 raft_consensus.cc:2454] T 3589a37edf7540afa56060a24ad7e381 P 1b6640624dd9445492ad7185990c1665 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7ef0d92a5e4c4b8482e1c52e9ed0b620 in term 0.
I20260925 02:44:00.551275 24777 raft_consensus.cc:2454] T 3589a37edf7540afa56060a24ad7e381 P 1b69d6babf064b30a674b864aee85903 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7ef0d92a5e4c4b8482e1c52e9ed0b620 in term 0.
I20260925 02:44:00.551704 24574 leader_election.cc:302] T 3589a37edf7540afa56060a24ad7e381 P 7ef0d92a5e4c4b8482e1c52e9ed0b620 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b6640624dd9445492ad7185990c1665, 7ef0d92a5e4c4b8482e1c52e9ed0b620; no voters:
I20260925 02:44:00.551993 24849 raft_consensus.cc:2781] T 3589a37edf7540afa56060a24ad7e381 P 7ef0d92a5e4c4b8482e1c52e9ed0b620 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 02:44:00.552090 24849 raft_consensus.cc:482] T 3589a37edf7540afa56060a24ad7e381 P 7ef0d92a5e4c4b8482e1c52e9ed0b620 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 02:44:00.552126 24849 raft_consensus.cc:3037] T 3589a37edf7540afa56060a24ad7e381 P 7ef0d92a5e4c4b8482e1c52e9ed0b620 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:44:00.553200 24849 raft_consensus.cc:504] T 3589a37edf7540afa56060a24ad7e381 P 7ef0d92a5e4c4b8482e1c52e9ed0b620 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b6640624dd9445492ad7185990c1665" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 34935 } } peers { permanent_uuid: "7ef0d92a5e4c4b8482e1c52e9ed0b620" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33565 } } peers { permanent_uuid: "1b69d6babf064b30a674b864aee85903" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 33603 } }
I20260925 02:44:00.553560 24849 leader_election.cc:288] T 3589a37edf7540afa56060a24ad7e381 P 7ef0d92a5e4c4b8482e1c52e9ed0b620 [CANDIDATE]: Term 1 election: Requested vote from peers 1b6640624dd9445492ad7185990c1665 (127.13.157.129:34935), 1b69d6babf064b30a674b864aee85903 (127.13.157.131:33603)
I20260925 02:44:00.553995 24506 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3589a37edf7540afa56060a24ad7e381" candidate_uuid: "7ef0d92a5e4c4b8482e1c52e9ed0b620" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b6640624dd9445492ad7185990c1665"
I20260925 02:44:00.553982 24777 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3589a37edf7540afa56060a24ad7e381" candidate_uuid: "7ef0d92a5e4c4b8482e1c52e9ed0b620" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b69d6babf064b30a674b864aee85903"
I20260925 02:44:00.554129 24777 raft_consensus.cc:3037] T 3589a37edf7540afa56060a24ad7e381 P 1b69d6babf064b30a674b864aee85903 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:44:00.554129 24506 raft_consensus.cc:3037] T 3589a37edf7540afa56060a24ad7e381 P 1b6640624dd9445492ad7185990c1665 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:44:00.555506 24777 raft_consensus.cc:2454] T 3589a37edf7540afa56060a24ad7e381 P 1b69d6babf064b30a674b864aee85903 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ef0d92a5e4c4b8482e1c52e9ed0b620 in term 1.
I20260925 02:44:00.555506 24506 raft_consensus.cc:2454] T 3589a37edf7540afa56060a24ad7e381 P 1b6640624dd9445492ad7185990c1665 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ef0d92a5e4c4b8482e1c52e9ed0b620 in term 1.
I20260925 02:44:00.555806 24575 leader_election.cc:302] T 3589a37edf7540afa56060a24ad7e381 P 7ef0d92a5e4c4b8482e1c52e9ed0b620 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b69d6babf064b30a674b864aee85903, 7ef0d92a5e4c4b8482e1c52e9ed0b620; no voters:
I20260925 02:44:00.555982 24849 raft_consensus.cc:2781] T 3589a37edf7540afa56060a24ad7e381 P 7ef0d92a5e4c4b8482e1c52e9ed0b620 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:44:00.556229 24849 raft_consensus.cc:686] T 3589a37edf7540afa56060a24ad7e381 P 7ef0d92a5e4c4b8482e1c52e9ed0b620 [term 1 LEADER]: Becoming Leader. State: Replica: 7ef0d92a5e4c4b8482e1c52e9ed0b620, State: Running, Role: LEADER
I20260925 02:44:00.556627 24849 consensus_queue.cc:237] T 3589a37edf7540afa56060a24ad7e381 P 7ef0d92a5e4c4b8482e1c52e9ed0b620 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b6640624dd9445492ad7185990c1665" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 34935 } } peers { permanent_uuid: "7ef0d92a5e4c4b8482e1c52e9ed0b620" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33565 } } peers { permanent_uuid: "1b69d6babf064b30a674b864aee85903" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 33603 } }
I20260925 02:44:00.560035 24360 catalog_manager.cc:5495] T 3589a37edf7540afa56060a24ad7e381 P 7ef0d92a5e4c4b8482e1c52e9ed0b620 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7ef0d92a5e4c4b8482e1c52e9ed0b620 (127.13.157.130). New cstate: current_term: 1 leader_uuid: "7ef0d92a5e4c4b8482e1c52e9ed0b620" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b6640624dd9445492ad7185990c1665" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 34935 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7ef0d92a5e4c4b8482e1c52e9ed0b620" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 33565 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1b69d6babf064b30a674b864aee85903" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 33603 } health_report { overall_health: UNKNOWN } } }
W20260925 02:44:00.568215 24689 tablet.cc:2369] T 3589a37edf7540afa56060a24ad7e381 P 7ef0d92a5e4c4b8482e1c52e9ed0b620: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 02:44:00.585850 24840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:44:00.582249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34706 (local address 127.13.157.130:33565)
0925 02:44:00.582425 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0925 02:44:00.582430 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:44:00.582447 (+ 17us) server_negotiation.cc:407] Connection header received
0925 02:44:00.582536 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:44:00.582538 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:44:00.582605 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:44:00.582698 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 02:44:00.582946 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:44:00.583494 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:44:00.584334 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:44:00.584576 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:44:00.584662 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:44:00.585017 (+ 355us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 02:44:00.585450 (+ 433us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 02:44:00.585557 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0925 02:44:00.585618 (+ 61us) server_negotiation.cc:299] Negotiation successful
0925 02:44:00.585670 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260925 02:44:00.591629 24777 raft_consensus.cc:1261] T 3589a37edf7540afa56060a24ad7e381 P 1b69d6babf064b30a674b864aee85903 [term 1 FOLLOWER]: Refusing update from remote peer 7ef0d92a5e4c4b8482e1c52e9ed0b620: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:44:00.591629 24506 raft_consensus.cc:1261] T 3589a37edf7540afa56060a24ad7e381 P 1b6640624dd9445492ad7185990c1665 [term 1 FOLLOWER]: Refusing update from remote peer 7ef0d92a5e4c4b8482e1c52e9ed0b620: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:44:00.592288 24851 consensus_queue.cc:1035] T 3589a37edf7540afa56060a24ad7e381 P 7ef0d92a5e4c4b8482e1c52e9ed0b620 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b69d6babf064b30a674b864aee85903" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 33603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:44:00.592471 24849 consensus_queue.cc:1035] T 3589a37edf7540afa56060a24ad7e381 P 7ef0d92a5e4c4b8482e1c52e9ed0b620 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6640624dd9445492ad7185990c1665" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 34935 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:44:00.602514 24856 mvcc.cc:205] Tried to move back new op lower bound from 7333086169456549888 to 7333086169323225088. Current Snapshot: MvccSnapshot[applied={T|T < 7333086169456549888}]
I20260925 02:44:00.608623 24360 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38148:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 02:44:00.608827 24360 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38148:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 02:44:00.611465 24360 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3556c13fa7cb40ad877870afc594123a: Sending DeleteTablet for 3 replicas of tablet 3589a37edf7540afa56060a24ad7e381
I20260925 02:44:00.612356 24486 tablet_service.cc:1514] Processing DeleteTablet for tablet 3589a37edf7540afa56060a24ad7e381 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:44:00 UTC) from {username='kudu', principal='kudu/127.13.157.190@KRBTEST.COM'} at 127.0.0.1:57114
I20260925 02:44:00.612380 24757 tablet_service.cc:1514] Processing DeleteTablet for tablet 3589a37edf7540afa56060a24ad7e381 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:44:00 UTC) from {username='kudu', principal='kudu/127.13.157.190@KRBTEST.COM'} at 127.0.0.1:52222
I20260925 02:44:00.612906 24863 tablet_replica.cc:330] T 3589a37edf7540afa56060a24ad7e381 P 1b6640624dd9445492ad7185990c1665: stopping tablet replica
I20260925 02:44:00.613058 24621 tablet_service.cc:1514] Processing DeleteTablet for tablet 3589a37edf7540afa56060a24ad7e381 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:44:00 UTC) from {username='kudu', principal='kudu/127.13.157.190@KRBTEST.COM'} at 127.0.0.1:34694
I20260925 02:44:00.613152 24864 tablet_replica.cc:330] T 3589a37edf7540afa56060a24ad7e381 P 1b69d6babf064b30a674b864aee85903: stopping tablet replica
I20260925 02:44:00.613508 24865 tablet_replica.cc:330] T 3589a37edf7540afa56060a24ad7e381 P 7ef0d92a5e4c4b8482e1c52e9ed0b620: stopping tablet replica
I20260925 02:44:00.613523 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 24423
I20260925 02:44:00.613821 24865 raft_consensus.cc:2229] T 3589a37edf7540afa56060a24ad7e381 P 7ef0d92a5e4c4b8482e1c52e9ed0b620 [term 1 LEADER]: Raft consensus shutting down.
I20260925 02:44:00.614048 24864 raft_consensus.cc:2229] T 3589a37edf7540afa56060a24ad7e381 P 1b69d6babf064b30a674b864aee85903 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 02:44:00.614588 24865 raft_consensus.cc:2258] T 3589a37edf7540afa56060a24ad7e381 P 7ef0d92a5e4c4b8482e1c52e9ed0b620 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:44:00.615098 24864 raft_consensus.cc:2258] T 3589a37edf7540afa56060a24ad7e381 P 1b69d6babf064b30a674b864aee85903 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:44:00.616412 24865 ts_tablet_manager.cc:1905] T 3589a37edf7540afa56060a24ad7e381 P 7ef0d92a5e4c4b8482e1c52e9ed0b620: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:44:00.617859 24864 ts_tablet_manager.cc:1905] T 3589a37edf7540afa56060a24ad7e381 P 1b69d6babf064b30a674b864aee85903: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:44:00.619455 24865 ts_tablet_manager.cc:1918] T 3589a37edf7540afa56060a24ad7e381 P 7ef0d92a5e4c4b8482e1c52e9ed0b620: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 02:44:00.619688 24865 log.cc:1196] T 3589a37edf7540afa56060a24ad7e381 P 7ef0d92a5e4c4b8482e1c52e9ed0b620: Deleting WAL directory at /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790304178325258-13942-0/minicluster-data/ts-1/wal/wals/3589a37edf7540afa56060a24ad7e381
I20260925 02:44:00.620097 24865 ts_tablet_manager.cc:1939] T 3589a37edf7540afa56060a24ad7e381 P 7ef0d92a5e4c4b8482e1c52e9ed0b620: Deleting consensus metadata
I20260925 02:44:00.620810 24864 ts_tablet_manager.cc:1918] T 3589a37edf7540afa56060a24ad7e381 P 1b69d6babf064b30a674b864aee85903: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 02:44:00.620915 24864 log.cc:1196] T 3589a37edf7540afa56060a24ad7e381 P 1b69d6babf064b30a674b864aee85903: Deleting WAL directory at /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790304178325258-13942-0/minicluster-data/ts-2/wal/wals/3589a37edf7540afa56060a24ad7e381
I20260925 02:44:00.621237 24864 ts_tablet_manager.cc:1939] T 3589a37edf7540afa56060a24ad7e381 P 1b69d6babf064b30a674b864aee85903: Deleting consensus metadata
I20260925 02:44:00.621665 24345 catalog_manager.cc:4841] TS 7ef0d92a5e4c4b8482e1c52e9ed0b620 (127.13.157.130:33565): tablet 3589a37edf7540afa56060a24ad7e381 (table test-table [id=c2148a15bac84c3eb8a457060a83b97b]) successfully deleted
W20260925 02:44:00.622071 24345 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.157.129:34935 (error 108)
I20260925 02:44:00.622311 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 24557
I20260925 02:44:00.622416 24346 catalog_manager.cc:4841] TS 1b69d6babf064b30a674b864aee85903 (127.13.157.131:33603): tablet 3589a37edf7540afa56060a24ad7e381 (table test-table [id=c2148a15bac84c3eb8a457060a83b97b]) successfully deleted
I20260925 02:44:00.624622 24838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:44:00.624277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.129:34935 (local address 127.0.0.1:57126)
0925 02:44:00.624430 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0925 02:44:00.624508 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.129:34935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260925 02:44:00.624841 24345 catalog_manager.cc:4584] TS 1b6640624dd9445492ad7185990c1665 (127.13.157.129:34935): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3589a37edf7540afa56060a24ad7e381: Network error: Client connection negotiation failed: client connection to 127.13.157.129:34935: connect: Connection refused (error 111)
I20260925 02:44:00.628516 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 24693
I20260925 02:44:00.633443 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 24328
2026-09-25T02:44:00Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2360 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 02:44:00 dist-test-slave-kznw krb5kdc[24869](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 02:44:00 dist-test-slave-kznw krb5kdc[24869](info): set up 2 sockets
Sep 25 02:44:00 dist-test-slave-kznw krb5kdc[24869](info): commencing operation
krb5kdc: starting...
W20260925 02:44:01.595302 13942 mini_kdc.cc:121] Time spent starting KDC: real 0.940s user 0.006s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/krb5kdc/test-user.keytab.
Sep 25 02:44:01 dist-test-slave-kznw krb5kdc[24869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304241, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T02:44:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T02:44:01Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.13.157.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.13.157.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.13.157.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/krb5kdc/rangeradmin_127.13.157.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.13.157.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/krb5kdc/rangeradmin_127.13.157.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.13.157.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.13.157.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.13.157.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/krb5kdc/rangerlookup_127.13.157.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.13.157.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/krb5kdc/rangerlookup_127.13.157.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.13.157.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.13.157.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.13.157.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/krb5kdc/HTTP_127.13.157.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.13.157.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/krb5kdc/HTTP_127.13.157.148@KRBTEST.COM.keytab.
I20260925 02:44:01.761976 13942 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-tasksiQ9fF/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/minicluster-data/postgres -l logfile start
2026-09-25 02:44:04.314 UTC [24899] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-25 02:44:04.314 UTC [24899] LOG: listening on IPv4 address "127.13.157.148", port 44837
2026-09-25 02:44:04.319 UTC [24899] LOG: listening on Unix socket "/tmp/.s.PGSQL.44837"
2026-09-25 02:44:04.340 UTC [24902] LOG: database system was shut down at 2026-09-25 02:44:02 UTC
2026-09-25 02:44:04.344 UTC [24899] LOG: database system is ready to accept connections
I20260925 02:44:05.219079 13942 mini_postgres.cc:96] Postgres bound to 44837
2026-09-25 02:44:05.225 UTC [24910] FATAL: database "slave" does not exist
127.13.157.148:44837 - accepting connections
I20260925 02:44:05.225823 13942 mini_ranger.cc:161] Starting Ranger...
I20260925 02:44:05.240163 13942 mini_ranger.cc:84] Created miniranger Postgres user
I20260925 02:44:05.753641 13942 mini_ranger.cc:87] Created ranger Postgres database
I20260925 02:44:05.753739 13942 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/minicluster-data/ranger-admin
2026-09-25 02:44:06,011 [I] DB FLAVOR :POSTGRES
2026-09-25 02:44:06,012 [I] --------- Verifying Ranger DB connection ---------
2026-09-25 02:44:06,012 [I] Checking connection..
2026-09-25 02:44:06,012 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksiQ9fF/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksiQ9fF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.157.148:44837/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-25 02:44:06,285 [I] Checking connection passed.
2026-09-25 02:44:06,286 [I] --------- Verifying version history table ---------
2026-09-25 02:44:06,286 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksiQ9fF/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksiQ9fF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.157.148:44837/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-25 02:44:06,515 [I] Table x_db_version_h does not exist in database ranger
2026-09-25 02:44:06,515 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksiQ9fF/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksiQ9fF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.157.148:44837/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-25 02:44:06,763 [I] Table x_db_version_h does not exist in database ranger
2026-09-25 02:44:06,764 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-25 02:44:06,764 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksiQ9fF/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksiQ9fF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.157.148:44837/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasksiQ9fF/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-25 02:44:06.999 UTC [24975] WARNING: there is no transaction in progress
2026-09-25 02:44:07,005 [I] create_dbversion_catalog.sql file imported successfully
2026-09-25 02:44:07,005 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksiQ9fF/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksiQ9fF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.157.148:44837/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-25 02:44:07,239 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-25 02:44:07,239 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-25 02:44:07,239 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksiQ9fF/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksiQ9fF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.157.148:44837/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-25 02:44:07,471 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksiQ9fF/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksiQ9fF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.157.148:44837/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-25 02:44:07,722 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksiQ9fF/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksiQ9fF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.157.148:44837/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-kznw.c.gcp-upstream.internal','N') ;"
2026-09-25 02:44:07,966 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksiQ9fF/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksiQ9fF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.157.148:44837/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-25 02:44:08,202 [I] Table x_portal_user does not exist in database ranger
2026-09-25 02:44:08,202 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksiQ9fF/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksiQ9fF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.157.148:44837/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-25 02:44:08,444 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-25 02:44:08,444 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksiQ9fF/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksiQ9fF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.157.148:44837/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-25 02:44:08,690 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksiQ9fF/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksiQ9fF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.157.148:44837/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-25 02:44:08,946 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-25 02:44:08,946 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksiQ9fF/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksiQ9fF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.157.148:44837/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasksiQ9fF/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-25 02:44:09.935 UTC [25083] WARNING: there is no transaction in progress
2026-09-25 02:44:09.952 UTC [25083] WARNING: there is no transaction in progress
2026-09-25 02:44:09.971 UTC [25083] WARNING: there is no transaction in progress
2026-09-25 02:44:09.988 UTC [25083] WARNING: there is no transaction in progress
2026-09-25 02:44:10.005 UTC [25083] WARNING: there is no transaction in progress
2026-09-25 02:44:10.025 UTC [25083] WARNING: there is no transaction in progress
2026-09-25 02:44:10.094 UTC [25083] WARNING: there is no transaction in progress
2026-09-25 02:44:10.102 UTC [25083] WARNING: there is no transaction in progress
2026-09-25 02:44:10.118 UTC [25083] WARNING: there is no transaction in progress
2026-09-25 02:44:10.129 UTC [25083] WARNING: there is no transaction in progress
2026-09-25 02:44:10.139 UTC [25083] WARNING: there is no transaction in progress
2026-09-25 02:44:10.154 UTC [25083] WARNING: there is no transaction in progress
2026-09-25 02:44:10.164 UTC [25083] WARNING: there is no transaction in progress
2026-09-25 02:44:10.170 UTC [25083] WARNING: there is no transaction in progress
2026-09-25 02:44:10.178 UTC [25083] WARNING: there is no transaction in progress
2026-09-25 02:44:10.186 UTC [25083] WARNING: there is no transaction in progress
2026-09-25 02:44:10,894 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-25 02:44:10,895 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksiQ9fF/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksiQ9fF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.157.148:44837/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-25 02:44:11,134 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-25 02:44:11,135 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksiQ9fF/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksiQ9fF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.157.148:44837/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-25 02:44:11,371 [I] Table x_portal_user already exists in database 'ranger'
2026-09-25 02:44:11,371 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksiQ9fF/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksiQ9fF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.157.148:44837/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-25 02:44:11,610 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-25 02:44:11,610 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksiQ9fF/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksiQ9fF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.157.148:44837/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-25 02:44:11,871 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksiQ9fF/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksiQ9fF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.157.148:44837/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-25 02:44:12,114 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksiQ9fF/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksiQ9fF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.157.148:44837/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-kznw.c.gcp-upstream.internal';"
2026-09-25 02:44:12,345 [I] CORE_DB_SCHEMA import status has been updated
2026-09-25 02:44:12,346 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksiQ9fF/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksiQ9fF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.157.148:44837/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-25 02:44:12,578 [I] DB_PATCHES have already been applied
I20260925 02:44:12.584579 13942 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/minicluster-data/ranger-admin:/tmp/dist-test-tasksiQ9fF/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-tasksiQ9fF/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-tasksiQ9fF/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksiQ9fF/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260925 02:44:12.584649 13942 mini_ranger.cc:193] Using host: 127.13.157.148
I20260925 02:44:12.588191 13942 mini_ranger.cc:227] Ranger admin URL: http://127.13.157.148:36347
Sep 25, 2026 2:44:12 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 25, 2026 2:44:12 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 25, 2026 2:44:13 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/minicluster-data/ranger-admin/ews/webapp
Sep 25, 2026 2:44:13 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 25, 2026 2:44:13 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 25, 2026 2:44:13 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/minicluster-data/ranger-admin/ews/webapp].
Sep 25, 2026 2:44:13 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260925 02:44:13.607468 13942 mini_ranger.cc:160] Time spent starting Ranger: real 8.382s user 0.009s sys 0.001s
Sep 25, 2026 2:44:33 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 25, 2026 2:44:33 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.XKeyREST
Sep 25, 2026 2:44:33 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 25, 2026 2:44:33 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 25, 2026 2:44:33 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 25, 2026 2:44:33 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 25, 2026 2:44:33 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 25, 2026 2:44:33 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 25, 2026 2:44:33 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 25, 2026 2:44:33 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 25, 2026 2:44:33 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 25, 2026 2:44:33 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 25, 2026 2:44:33 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 25, 2026 2:44:33 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 25, 2026 2:44:33 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 25, 2026 2:44:33 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 25, 2026 2:44:33 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 25, 2026 2:44:33 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260925 02:44:34.510385 13942 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.13.157.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.13.157.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.13.157.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/krb5kdc/rangerkms_127.13.157.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.13.157.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/krb5kdc/rangerkms_127.13.157.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.13.157.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.13.157.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.13.157.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/krb5kdc/HTTP_127.13.157.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.13.157.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/krb5kdc/HTTP_127.13.157.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 25 02:44:34 dist-test-slave-kznw krb5kdc[24869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304274, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260925 02:44:34.603354 13942 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260925 02:44:34.612808 13942 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260925 02:44:35.476804 13942 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260925 02:44:35.476895 13942 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/minicluster-data/ranger-kms
I20260925 02:44:35.477005 13942 mini_ranger_kms.cc:227] Using postgres at 127.13.157.148:44837
2026-09-25 02:44:36,093 [I] DB FLAVOR :POSTGRES
2026-09-25 02:44:36,093 [I] --------- Verifying Ranger DB connection ---------
2026-09-25 02:44:36,093 [I] Checking connection
2026-09-25 02:44:36,093 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksiQ9fF/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.157.148:44837/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-25 02:44:36,355 [I] connection success
2026-09-25 02:44:36,355 [I] --------- Verifying Ranger DB tables ---------
2026-09-25 02:44:36,355 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-25 02:44:36,355 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksiQ9fF/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.157.148:44837/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-25 02:44:36,585 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-25 02:44:36,585 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-25 02:44:36,585 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-25 02:44:36,586 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksiQ9fF/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.157.148:44837/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-25 02:44:36,844 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260925 02:44:37.291260 13942 mini_ranger_kms.cc:310] Created kms service
I20260925 02:44:37.486502 13942 mini_ranger_kms.cc:326] Created kudu user
I20260925 02:44:37.527942 13942 mini_ranger_kms.cc:343] Created rangerkms user
Sep 25 02:44:37 dist-test-slave-kznw krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304274, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.13.157.148@KRBTEST.COM
I20260925 02:44:37.877722 13942 mini_ranger_kms.cc:384] Added ranger policy
I20260925 02:44:37.877883 13942 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/minicluster-data/ranger-kms:/tmp/dist-test-tasksiQ9fF/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksiQ9fF/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-tasksiQ9fF/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260925 02:44:37.877907 13942 mini_ranger_kms.cc:240] Using host: 127.13.157.148
I20260925 02:44:37.881894 13942 mini_ranger_kms.cc:276] Ranger KMS PID: 25264
I20260925 02:44:37.882009 13942 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.13.157.148:49187
Sep 25, 2026 2:44:39 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 25, 2026 2:44:39 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 25, 2026 2:44:39 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 25, 2026 2:44:39 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 25, 2026 2:44:39 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/minicluster-data/ranger-kms/ews/webapp].
Sep 25, 2026 2:44:39 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260925 02:44:39.955885 13942 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.353s user 0.013s sys 0.220s
I20260925 02:44:39.956017 13942 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260925 02:44:39.956117 13942 mini_ranger_kms.cc:401] 127.13.157.148:49187/kms/v1/keys
Sep 25 02:44:40 dist-test-slave-kznw krb5kdc[24869](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790304280, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.13.157.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 25 02:44:40 dist-test-slave-kznw krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304280, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.13.157.148@KRBTEST.COM for HTTP/127.13.157.148@KRBTEST.COM
Sep 25 02:44:40 dist-test-slave-kznw krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304280, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.13.157.148@KRBTEST.COM for HTTP/127.13.157.148@KRBTEST.COM
Sep 25 02:44:43 dist-test-slave-kznw krb5kdc[24869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304283, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.13.157.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.157.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.157.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab.
Entry for principal kudu/127.13.157.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab.
I20260925 02:44:43.183591 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.157.190:46205
--webserver_interface=127.13.157.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab
--principal=kudu/127.13.157.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:35115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.157.190:46205
--ranger_config_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.157.148:49187/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:44:43.295236 25316 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:44:43.295516 25316 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:44:43.295555 25316 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:44:43.299638 25316 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:44:43.299695 25316 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:44:43.299712 25316 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:44:43.299729 25316 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:44:43.299758 25316 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:44:43.305207 25316 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:35115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/minicluster-data/master-0/wal
--ranger_kms_url=127.13.157.148:49187/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.13.157.190:46205
--ranger_config_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab
--principal=kudu/127.13.157.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.157.190:46205
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.157.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25316
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:44:43.306331 25316 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:44:43.307412 25316 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 02:44:43.316100 25316 server_base.cc:1034] running on GCE node
W20260925 02:44:43.316187 25322 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:44:43.316118 25321 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:44:43.316362 25324 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:44:43.317001 25316 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:44:43.318120 25316 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:44:43.319284 25316 hybrid_clock.cc:648] HybridClock initialized: now 1790304283319271 us; error 28 us; skew 500 ppm
Sep 25 02:44:43 dist-test-slave-kznw krb5kdc[24869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304283, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:44:43.322300 25316 init.cc:377] Logged in from keytab as kudu/127.13.157.190@KRBTEST.COM (short username kudu)
I20260925 02:44:43.323271 25316 webserver.cc:460] Webserver started at http://127.13.157.190:42343/ using document root <none> and password file <none>
I20260925 02:44:43.323791 25316 fs_manager.cc:359] Metadata directory not provided
I20260925 02:44:43.323841 25316 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:44:43.324021 25316 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 25 02:44:43 dist-test-slave-kznw krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304283, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.190@KRBTEST.COM for HTTP/127.13.157.148@KRBTEST.COM
I20260925 02:44:43.388047 25316 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/minicluster-data/master-0/data/instance:
uuid: "cd798d439007493a9e7bd4fa5dbef9f7"
format_stamp: "Formatted at 2026-09-25 02:44:43 on dist-test-slave-kznw"
server_key: "da715562bd374a7a28080af6a39d7e39"
server_key_iv: "d8a35159195d2abb7cf392275cfa3564"
server_key_version: "kuduclusterkey@0"
I20260925 02:44:43.388665 25316 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/minicluster-data/master-0/wal/instance:
uuid: "cd798d439007493a9e7bd4fa5dbef9f7"
format_stamp: "Formatted at 2026-09-25 02:44:43 on dist-test-slave-kznw"
server_key: "da715562bd374a7a28080af6a39d7e39"
server_key_iv: "d8a35159195d2abb7cf392275cfa3564"
server_key_version: "kuduclusterkey@0"
I20260925 02:44:43.393395 25316 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260925 02:44:43.412331 25331 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:44:43.414173 25316 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 02:44:43.414328 25316 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/minicluster-data/master-0/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/minicluster-data/master-0/wal
uuid: "cd798d439007493a9e7bd4fa5dbef9f7"
format_stamp: "Formatted at 2026-09-25 02:44:43 on dist-test-slave-kznw"
server_key: "da715562bd374a7a28080af6a39d7e39"
server_key_iv: "d8a35159195d2abb7cf392275cfa3564"
server_key_version: "kuduclusterkey@0"
I20260925 02:44:43.414487 25316 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:44:43.437109 25316 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:44:43.440411 25316 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:44:43.440619 25316 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:44:43.449419 25384 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.190:46205 every 8 connection(s)
I20260925 02:44:43.449416 25316 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.190:46205
I20260925 02:44:43.450563 25316 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
I20260925 02:44:43.451208 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 25316
I20260925 02:44:43.451318 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/minicluster-data/master-0/wal/instance
I20260925 02:44:43.451588 13942 external_mini_cluster.cc:1428] Setting key f05b7f48971d6050022220dc89b75413
I20260925 02:44:43.453512 25385 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:44:43.459969 25385 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd798d439007493a9e7bd4fa5dbef9f7: Bootstrap starting.
Sep 25 02:44:43 dist-test-slave-kznw krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304283, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.157.190@KRBTEST.COM
I20260925 02:44:43.462218 25385 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cd798d439007493a9e7bd4fa5dbef9f7: Neither blocks nor log segments found. Creating new log.
I20260925 02:44:43.463354 25385 log.cc:824] T 00000000000000000000000000000000 P cd798d439007493a9e7bd4fa5dbef9f7: Log is configured to *not* fsync() on all Append() calls
I20260925 02:44:43.465301 25385 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd798d439007493a9e7bd4fa5dbef9f7: No bootstrap required, opened a new log
I20260925 02:44:43.465540 25388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:44:43.452998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37126 (local address 127.13.157.190:46205)
0925 02:44:43.453417 (+ 419us) server_negotiation.cc:206] Beginning negotiation
0925 02:44:43.453434 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0925 02:44:43.453470 (+ 36us) server_negotiation.cc:407] Connection header received
0925 02:44:43.454129 (+ 659us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:44:43.454146 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:44:43.454495 (+ 349us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:44:43.454790 (+ 295us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:44:43.455683 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:44:43.456667 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:44:43.457780 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:44:43.458297 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:44:43.458418 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:44:43.461108 (+ 2690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:44:43.461114 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:44:43.461127 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:44:43.461153 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:44:43.462720 (+ 1567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:44:43.463198 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:44:43.463203 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:44:43.463208 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:44:43.463310 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:44:43.463582 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:44:43.463586 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:44:43.463587 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:44:43.464078 (+ 491us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:44:43.464258 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0925 02:44:43.464538 (+ 280us) server_negotiation.cc:299] Negotiation successful
0925 02:44:43.464801 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":127,"threads_started":1}
I20260925 02:44:43.468683 25385 raft_consensus.cc:348] T 00000000000000000000000000000000 P cd798d439007493a9e7bd4fa5dbef9f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd798d439007493a9e7bd4fa5dbef9f7" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 46205 } }
I20260925 02:44:43.468914 25385 raft_consensus.cc:374] T 00000000000000000000000000000000 P cd798d439007493a9e7bd4fa5dbef9f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:44:43.468983 25385 raft_consensus.cc:729] T 00000000000000000000000000000000 P cd798d439007493a9e7bd4fa5dbef9f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd798d439007493a9e7bd4fa5dbef9f7, State: Initialized, Role: FOLLOWER
I20260925 02:44:43.469436 25385 consensus_queue.cc:260] T 00000000000000000000000000000000 P cd798d439007493a9e7bd4fa5dbef9f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd798d439007493a9e7bd4fa5dbef9f7" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 46205 } }
I20260925 02:44:43.469575 25385 raft_consensus.cc:388] T 00000000000000000000000000000000 P cd798d439007493a9e7bd4fa5dbef9f7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 02:44:43.469673 25385 raft_consensus.cc:482] T 00000000000000000000000000000000 P cd798d439007493a9e7bd4fa5dbef9f7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 02:44:43.469777 25385 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cd798d439007493a9e7bd4fa5dbef9f7 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:44:43.470649 25385 raft_consensus.cc:504] T 00000000000000000000000000000000 P cd798d439007493a9e7bd4fa5dbef9f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd798d439007493a9e7bd4fa5dbef9f7" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 46205 } }
I20260925 02:44:43.470999 25385 leader_election.cc:302] T 00000000000000000000000000000000 P cd798d439007493a9e7bd4fa5dbef9f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cd798d439007493a9e7bd4fa5dbef9f7; no voters:
I20260925 02:44:43.471299 25385 leader_election.cc:288] T 00000000000000000000000000000000 P cd798d439007493a9e7bd4fa5dbef9f7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 02:44:43.471450 25390 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cd798d439007493a9e7bd4fa5dbef9f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:44:43.471673 25390 raft_consensus.cc:686] T 00000000000000000000000000000000 P cd798d439007493a9e7bd4fa5dbef9f7 [term 1 LEADER]: Becoming Leader. State: Replica: cd798d439007493a9e7bd4fa5dbef9f7, State: Running, Role: LEADER
I20260925 02:44:43.471974 25390 consensus_queue.cc:237] T 00000000000000000000000000000000 P cd798d439007493a9e7bd4fa5dbef9f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd798d439007493a9e7bd4fa5dbef9f7" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 46205 } }
I20260925 02:44:43.472479 25385 sys_catalog.cc:564] T 00000000000000000000000000000000 P cd798d439007493a9e7bd4fa5dbef9f7 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:44:43.473441 25391 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd798d439007493a9e7bd4fa5dbef9f7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cd798d439007493a9e7bd4fa5dbef9f7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd798d439007493a9e7bd4fa5dbef9f7" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 46205 } } }
I20260925 02:44:43.473567 25391 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd798d439007493a9e7bd4fa5dbef9f7 [sys.catalog]: This master's current role is: LEADER
I20260925 02:44:43.473831 25392 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd798d439007493a9e7bd4fa5dbef9f7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cd798d439007493a9e7bd4fa5dbef9f7. Latest consensus state: current_term: 1 leader_uuid: "cd798d439007493a9e7bd4fa5dbef9f7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd798d439007493a9e7bd4fa5dbef9f7" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 46205 } } }
I20260925 02:44:43.473929 25392 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd798d439007493a9e7bd4fa5dbef9f7 [sys.catalog]: This master's current role is: LEADER
I20260925 02:44:43.474020 25399 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:44:43.476897 25399 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:44:43.477353 25385 ranger_client.cc:318] Using new properties file: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260925 02:44:43.487311 25399 catalog_manager.cc:1347] Generated new cluster ID: 80c132fc211b4132bc51dd6c0fd652c5
I20260925 02:44:43.487404 25399 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 02:44:43.507474 25399 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 02:44:43.509100 25399 catalog_manager.cc:1504] Loading token signing keys...
I20260925 02:44:43.515609 25399 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cd798d439007493a9e7bd4fa5dbef9f7: Generated new TSK 0
I20260925 02:44:43.516501 25399 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.157.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.157.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.157.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.129.keytab.
Entry for principal kudu/127.13.157.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.129.keytab.
I20260925 02:44:44.215561 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.157.148:49187/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.157.129:0
--local_ip_for_outbound_sockets=127.13.157.129
--webserver_interface=127.13.157.129
--webserver_port=0
--tserver_master_addrs=127.13.157.190:46205
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.129.keytab
--principal=kudu/127.13.157.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:35115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:44:44.332623 25431 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:44:44.332897 25431 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:44:44.332960 25431 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:44:44.337087 25431 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:44:44.337160 25431 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:44:44.337244 25431 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.157.129
I20260925 02:44:44.342763 25431 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:35115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.13.157.148:49187/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.129.keytab
--principal=kudu/127.13.157.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.157.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.157.129
--webserver_port=0
--tserver_master_addrs=127.13.157.190:46205
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25431
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.157.129
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:44:44.343923 25431 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:44:44.344969 25431 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:44:44.354777 25440 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:44:44.354825 25438 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:44:44.354789 25437 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:44:44.355090 25431 server_base.cc:1034] running on GCE node
I20260925 02:44:44.355611 25431 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:44:44.356307 25431 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:44:44.357486 25431 hybrid_clock.cc:648] HybridClock initialized: now 1790304284357469 us; error 34 us; skew 500 ppm
Sep 25 02:44:44 dist-test-slave-kznw krb5kdc[24869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304284, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:44:44.360479 25431 init.cc:377] Logged in from keytab as kudu/127.13.157.129@KRBTEST.COM (short username kudu)
I20260925 02:44:44.361450 25431 webserver.cc:460] Webserver started at http://127.13.157.129:45121/ using document root <none> and password file <none>
I20260925 02:44:44.362001 25431 fs_manager.cc:359] Metadata directory not provided
I20260925 02:44:44.362062 25431 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:44:44.362249 25431 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 25 02:44:44 dist-test-slave-kznw krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304284, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.129@KRBTEST.COM for HTTP/127.13.157.148@KRBTEST.COM
I20260925 02:44:44.379112 25431 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/minicluster-data/ts-0/data/instance:
uuid: "fcf328843180471eb5d2020eb5237979"
format_stamp: "Formatted at 2026-09-25 02:44:44 on dist-test-slave-kznw"
server_key: "c2fcd4bca0edaedbe00a41c4acbf09d0"
server_key_iv: "8bdab1b5ea380cc55c92a4a8d5739c49"
server_key_version: "kuduclusterkey@0"
I20260925 02:44:44.379783 25431 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/minicluster-data/ts-0/wal/instance:
uuid: "fcf328843180471eb5d2020eb5237979"
format_stamp: "Formatted at 2026-09-25 02:44:44 on dist-test-slave-kznw"
server_key: "c2fcd4bca0edaedbe00a41c4acbf09d0"
server_key_iv: "8bdab1b5ea380cc55c92a4a8d5739c49"
server_key_version: "kuduclusterkey@0"
I20260925 02:44:44.384141 25431 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260925 02:44:44.411271 25449 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:44:44.412806 25431 fs_manager.cc:725] Time spent opening block manager: real 0.009s user 0.002s sys 0.000s
I20260925 02:44:44.412973 25431 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/minicluster-data/ts-0/wal
uuid: "fcf328843180471eb5d2020eb5237979"
format_stamp: "Formatted at 2026-09-25 02:44:44 on dist-test-slave-kznw"
server_key: "c2fcd4bca0edaedbe00a41c4acbf09d0"
server_key_iv: "8bdab1b5ea380cc55c92a4a8d5739c49"
server_key_version: "kuduclusterkey@0"
I20260925 02:44:44.413106 25431 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:44:44.431461 25431 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:44:44.435128 25431 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:44:44.435371 25431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:44:44.436131 25431 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:44:44.437234 25431 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:44:44.437285 25431 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:44:44.437340 25431 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:44:44.437379 25431 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260925 02:44:44.447791 25431 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.129:36939
I20260925 02:44:44.448029 25565 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.129:36939 every 8 connection(s)
I20260925 02:44:44.449102 25431 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/minicluster-data/ts-0/data/info.pb
I20260925 02:44:44.458715 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 25431
I20260925 02:44:44.458868 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/minicluster-data/ts-0/wal/instance
I20260925 02:44:44.459230 13942 external_mini_cluster.cc:1428] Setting key e8d6fe968ac784f1ca206bee869523fa
Sep 25 02:44:44 dist-test-slave-kznw krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304284, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.129@KRBTEST.COM for kudu/127.13.157.190@KRBTEST.COM
I20260925 02:44:44.473781 25569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:44:44.453357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.129:36047 (local address 127.13.157.190:46205)
0925 02:44:44.455240 (+ 1883us) server_negotiation.cc:206] Beginning negotiation
0925 02:44:44.455249 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 02:44:44.455264 (+ 15us) server_negotiation.cc:407] Connection header received
0925 02:44:44.455303 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:44:44.455306 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:44:44.455353 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:44:44.457000 (+ 1647us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:44:44.457119 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:44:44.457952 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:44:44.460135 (+ 2183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:44:44.460490 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:44:44.463339 (+ 2849us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:44:44.466126 (+ 2787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:44:44.466130 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:44:44.466133 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:44:44.466162 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:44:44.468032 (+ 1870us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:44:44.468664 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:44:44.468668 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:44:44.468670 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:44:44.468766 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:44:44.469176 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:44:44.469180 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:44:44.469182 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:44:44.469416 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:44:44.469499 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0925 02:44:44.473339 (+ 3840us) server_negotiation.cc:299] Negotiation successful
0925 02:44:44.473582 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1803,"thread_start_us":83,"threads_started":1}
WARNING: no policy specified for kudu/127.13.157.130@KRBTEST.COM; defaulting to no policy
I20260925 02:44:44.479437 25568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:44:44.452234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:46205 (local address 127.13.157.129:36047)
0925 02:44:44.453610 (+ 1376us) negotiation.cc:107] Waiting for socket to connect
0925 02:44:44.453669 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0925 02:44:44.454770 (+ 1101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:44:44.455737 (+ 967us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:44:44.455756 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:44:44.456122 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:44:44.456833 (+ 711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:44:44.456856 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:44:44.458101 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:44:44.458105 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:44:44.459993 (+ 1888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:44:44.460005 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:44:44.462833 (+ 2828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:44:44.462838 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:44:44.463085 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:44:44.464103 (+ 1018us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:44:44.464126 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:44:44.465888 (+ 1762us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:44:44.468182 (+ 2294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:44:44.468188 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:44:44.468204 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:44:44.468545 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:44:44.468875 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:44:44.468878 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:44:44.468892 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:44:44.469065 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:44:44.469567 (+ 502us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:44:44.469574 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:44:44.469909 (+ 335us) client_negotiation.cc:769] Sending connection context
0925 02:44:44.473979 (+ 4070us) client_negotiation.cc:240] Negotiation successful
0925 02:44:44.478221 (+ 4242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1142,"thread_start_us":142,"threads_started":1}
I20260925 02:44:44.481045 25566 heartbeater.cc:347] Connected to a master server at 127.13.157.190:46205
I20260925 02:44:44.481312 25566 heartbeater.cc:464] Registering TS with master...
I20260925 02:44:44.481931 25566 heartbeater.cc:511] Master 127.13.157.190:46205 requested a full tablet report, sending...
I20260925 02:44:44.483896 25348 ts_manager.cc:194] Registered new tserver with Master: fcf328843180471eb5d2020eb5237979 (127.13.157.129:36939)
I20260925 02:44:44.485534 25348 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.157.129@KRBTEST.COM'} at 127.13.157.129:36047
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.157.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.157.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.130.keytab.
Entry for principal kudu/127.13.157.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.130.keytab.
I20260925 02:44:44.524636 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.157.148:49187/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.157.130:0
--local_ip_for_outbound_sockets=127.13.157.130
--webserver_interface=127.13.157.130
--webserver_port=0
--tserver_master_addrs=127.13.157.190:46205
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.130.keytab
--principal=kudu/127.13.157.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:35115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 25 02:44:44 dist-test-slave-kznw krb5kdc[24869](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790304284, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260925 02:44:44.693397 25574 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:44:44.693877 25574 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:44:44.693990 25574 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:44:44.700147 25574 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:44:44.700304 25574 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:44:44.700433 25574 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.157.130
I20260925 02:44:44.708221 25574 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:35115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.13.157.148:49187/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.130.keytab
--principal=kudu/127.13.157.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.157.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.157.130
--webserver_port=0
--tserver_master_addrs=127.13.157.190:46205
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25574
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.157.130
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:44:44.709803 25574 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:44:44.711186 25574 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:44:44.723779 25581 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:44:44.723912 25574 server_base.cc:1034] running on GCE node
W20260925 02:44:44.724226 25583 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:44:44.724272 25580 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:44:44.724843 25574 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:44:44.725626 25574 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:44:44.726887 25574 hybrid_clock.cc:648] HybridClock initialized: now 1790304284726843 us; error 48 us; skew 500 ppm
Sep 25 02:44:44 dist-test-slave-kznw krb5kdc[24869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304284, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:44:44.730661 25574 init.cc:377] Logged in from keytab as kudu/127.13.157.130@KRBTEST.COM (short username kudu)
I20260925 02:44:44.731806 25574 webserver.cc:460] Webserver started at http://127.13.157.130:45281/ using document root <none> and password file <none>
I20260925 02:44:44.732544 25574 fs_manager.cc:359] Metadata directory not provided
I20260925 02:44:44.732631 25574 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:44:44.732851 25574 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 25 02:44:44 dist-test-slave-kznw krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304284, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.130@KRBTEST.COM for HTTP/127.13.157.148@KRBTEST.COM
I20260925 02:44:44.749298 25574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/minicluster-data/ts-1/data/instance:
uuid: "8bb07daa6a7d48a1b6eb198cda727f72"
format_stamp: "Formatted at 2026-09-25 02:44:44 on dist-test-slave-kznw"
server_key: "b7340e1b1450944e0f1b8d81e0b13d64"
server_key_iv: "0440e590f60fd0fddc93563acd4c140c"
server_key_version: "kuduclusterkey@0"
I20260925 02:44:44.750216 25574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/minicluster-data/ts-1/wal/instance:
uuid: "8bb07daa6a7d48a1b6eb198cda727f72"
format_stamp: "Formatted at 2026-09-25 02:44:44 on dist-test-slave-kznw"
server_key: "b7340e1b1450944e0f1b8d81e0b13d64"
server_key_iv: "0440e590f60fd0fddc93563acd4c140c"
server_key_version: "kuduclusterkey@0"
I20260925 02:44:44.755924 25574 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260925 02:44:44.778105 25590 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:44:44.780194 25574 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.002s sys 0.000s
I20260925 02:44:44.780431 25574 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/minicluster-data/ts-1/wal
uuid: "8bb07daa6a7d48a1b6eb198cda727f72"
format_stamp: "Formatted at 2026-09-25 02:44:44 on dist-test-slave-kznw"
server_key: "b7340e1b1450944e0f1b8d81e0b13d64"
server_key_iv: "0440e590f60fd0fddc93563acd4c140c"
server_key_version: "kuduclusterkey@0"
I20260925 02:44:44.780584 25574 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:44:44.796944 25574 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:44:44.800729 25574 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:44:44.801003 25574 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:44:44.801836 25574 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:44:44.803006 25574 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:44:44.803108 25574 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:44:44.803208 25574 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:44:44.803313 25574 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:44:44.813587 25574 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.130:35321
I20260925 02:44:44.814944 25574 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/minicluster-data/ts-1/data/info.pb
I20260925 02:44:44.823441 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 25574
I20260925 02:44:44.823590 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/minicluster-data/ts-1/wal/instance
I20260925 02:44:44.823980 13942 external_mini_cluster.cc:1428] Setting key 9d1e24313e7abe642531a7abca9b174e
I20260925 02:44:44.824530 25704 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.130:35321 every 8 connection(s)
Sep 25 02:44:44 dist-test-slave-kznw krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304284, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.130@KRBTEST.COM for kudu/127.13.157.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.157.131@KRBTEST.COM; defaulting to no policy
I20260925 02:44:44.855837 25569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:44:44.834647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.130:36349 (local address 127.13.157.190:46205)
0925 02:44:44.834778 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0925 02:44:44.834782 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:44:44.835257 (+ 475us) server_negotiation.cc:407] Connection header received
0925 02:44:44.836532 (+ 1275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:44:44.836536 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:44:44.836592 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:44:44.836644 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:44:44.838320 (+ 1676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:44:44.839208 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:44:44.840946 (+ 1738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:44:44.841311 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:44:44.841427 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:44:44.851058 (+ 9631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:44:44.851063 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:44:44.851067 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:44:44.851108 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:44:44.852852 (+ 1744us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:44:44.853701 (+ 849us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:44:44.853705 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:44:44.853707 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:44:44.853801 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:44:44.854310 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:44:44.854313 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:44:44.854315 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:44:44.854591 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:44:44.854663 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0925 02:44:44.855513 (+ 850us) server_negotiation.cc:299] Negotiation successful
0925 02:44:44.855658 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260925 02:44:44.857398 25708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:44:44.833439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:46205 (local address 127.13.157.130:36349)
0925 02:44:44.835093 (+ 1654us) negotiation.cc:107] Waiting for socket to connect
0925 02:44:44.835156 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0925 02:44:44.836254 (+ 1098us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:44:44.837063 (+ 809us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:44:44.837072 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:44:44.837459 (+ 387us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:44:44.838139 (+ 680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:44:44.838160 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:44:44.839371 (+ 1211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:44:44.839374 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:44:44.840805 (+ 1431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:44:44.840815 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:44:44.841653 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:44:44.841657 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:44:44.841862 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:44:44.844027 (+ 2165us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:44:44.844049 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:44:44.845871 (+ 1822us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:44:44.853022 (+ 7151us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:44:44.853030 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:44:44.853055 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:44:44.853572 (+ 517us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:44:44.853976 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:44:44.853979 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:44:44.853981 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:44:44.854194 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:44:44.854748 (+ 554us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:44:44.854756 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:44:44.855203 (+ 447us) client_negotiation.cc:769] Sending connection context
0925 02:44:44.856293 (+ 1090us) client_negotiation.cc:240] Negotiation successful
0925 02:44:44.856594 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1394,"thread_start_us":176,"threads_started":1}
I20260925 02:44:44.858563 25705 heartbeater.cc:347] Connected to a master server at 127.13.157.190:46205
I20260925 02:44:44.858804 25705 heartbeater.cc:464] Registering TS with master...
I20260925 02:44:44.859437 25705 heartbeater.cc:511] Master 127.13.157.190:46205 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.157.131@KRBTEST.COM" created.
I20260925 02:44:44.860755 25348 ts_manager.cc:194] Registered new tserver with Master: 8bb07daa6a7d48a1b6eb198cda727f72 (127.13.157.130:35321)
I20260925 02:44:44.861454 25348 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.157.130@KRBTEST.COM'} at 127.13.157.130:36349
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.157.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.131.keytab.
Entry for principal kudu/127.13.157.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.131.keytab.
I20260925 02:44:44.882586 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.157.148:49187/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.157.131:0
--local_ip_for_outbound_sockets=127.13.157.131
--webserver_interface=127.13.157.131
--webserver_port=0
--tserver_master_addrs=127.13.157.190:46205
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.131.keytab
--principal=kudu/127.13.157.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:35115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:44:45.053509 25710 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:44:45.053896 25710 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:44:45.054015 25710 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:44:45.060036 25710 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:44:45.060220 25710 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:44:45.060371 25710 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.157.131
I20260925 02:44:45.068641 25710 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:35115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.13.157.148:49187/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.131.keytab
--principal=kudu/127.13.157.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.157.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.157.131
--webserver_port=0
--tserver_master_addrs=127.13.157.190:46205
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25710
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.157.131
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:44:45.070138 25710 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:44:45.071470 25710 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:44:45.083791 25716 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:44:45.083904 25718 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:44:45.084587 25715 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:44:45.085119 25710 server_base.cc:1034] running on GCE node
I20260925 02:44:45.085740 25710 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:44:45.086584 25710 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:44:45.087782 25710 hybrid_clock.cc:648] HybridClock initialized: now 1790304285087735 us; error 60 us; skew 500 ppm
Sep 25 02:44:45 dist-test-slave-kznw krb5kdc[24869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304285, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:44:45.091460 25710 init.cc:377] Logged in from keytab as kudu/127.13.157.131@KRBTEST.COM (short username kudu)
I20260925 02:44:45.092648 25710 webserver.cc:460] Webserver started at http://127.13.157.131:44507/ using document root <none> and password file <none>
I20260925 02:44:45.093395 25710 fs_manager.cc:359] Metadata directory not provided
I20260925 02:44:45.093494 25710 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:44:45.093815 25710 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 25 02:44:45 dist-test-slave-kznw krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304285, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.131@KRBTEST.COM for HTTP/127.13.157.148@KRBTEST.COM
I20260925 02:44:45.110146 25710 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/minicluster-data/ts-2/data/instance:
uuid: "a4aa579542654e54bef80d5e41c47e0a"
format_stamp: "Formatted at 2026-09-25 02:44:45 on dist-test-slave-kznw"
server_key: "aedabdcf8ae4cea83e29150f3505141c"
server_key_iv: "fc3fb4691dd0055dcea24720e6a0940d"
server_key_version: "kuduclusterkey@0"
I20260925 02:44:45.111770 25710 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/minicluster-data/ts-2/wal/instance:
uuid: "a4aa579542654e54bef80d5e41c47e0a"
format_stamp: "Formatted at 2026-09-25 02:44:45 on dist-test-slave-kznw"
server_key: "aedabdcf8ae4cea83e29150f3505141c"
server_key_iv: "fc3fb4691dd0055dcea24720e6a0940d"
server_key_version: "kuduclusterkey@0"
I20260925 02:44:45.120532 25710 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.000s sys 0.004s
I20260925 02:44:45.139391 25725 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:44:45.141183 25710 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 02:44:45.141326 25710 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/minicluster-data/ts-2/wal
uuid: "a4aa579542654e54bef80d5e41c47e0a"
format_stamp: "Formatted at 2026-09-25 02:44:45 on dist-test-slave-kznw"
server_key: "aedabdcf8ae4cea83e29150f3505141c"
server_key_iv: "fc3fb4691dd0055dcea24720e6a0940d"
server_key_version: "kuduclusterkey@0"
I20260925 02:44:45.141456 25710 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
Sep 25 02:44:45 dist-test-slave-kznw krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304284, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.190@KRBTEST.COM for HTTP/127.13.157.148@KRBTEST.COM
I20260925 02:44:45.163228 25710 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:44:45.167620 25710 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:44:45.167829 25710 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:44:45.168530 25710 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:44:45.169600 25710 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:44:45.169672 25710 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:44:45.169780 25710 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:44:45.169833 25710 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:44:45.180820 25710 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.131:37883
I20260925 02:44:45.182094 25710 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/minicluster-data/ts-2/data/info.pb
I20260925 02:44:45.184613 25840 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.131:37883 every 8 connection(s)
I20260925 02:44:45.189738 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 25710
I20260925 02:44:45.189873 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/minicluster-data/ts-2/wal/instance
I20260925 02:44:45.190193 13942 external_mini_cluster.cc:1428] Setting key 84f097e5a0cee48214033f251f2f3e36
Sep 25 02:44:45 dist-test-slave-kznw krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304285, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.131@KRBTEST.COM for kudu/127.13.157.190@KRBTEST.COM
I20260925 02:44:45.204839 25569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:44:45.186050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.131:39979 (local address 127.13.157.190:46205)
0925 02:44:45.186163 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0925 02:44:45.186167 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:44:45.186358 (+ 191us) server_negotiation.cc:407] Connection header received
0925 02:44:45.187385 (+ 1027us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:44:45.187389 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:44:45.187446 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:44:45.187511 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:44:45.192199 (+ 4688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:44:45.192891 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:44:45.194448 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:44:45.194776 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:44:45.194872 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:44:45.197763 (+ 2891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:44:45.197767 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:44:45.197770 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:44:45.197796 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:44:45.200349 (+ 2553us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:44:45.201174 (+ 825us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:44:45.201178 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:44:45.201180 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:44:45.201292 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:44:45.201722 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:44:45.201726 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:44:45.201728 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:44:45.201955 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:44:45.202037 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0925 02:44:45.204594 (+ 2557us) server_negotiation.cc:299] Negotiation successful
0925 02:44:45.204715 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260925 02:44:45.206647 25843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:44:45.184857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:46205 (local address 127.13.157.131:39979)
0925 02:44:45.186226 (+ 1369us) negotiation.cc:107] Waiting for socket to connect
0925 02:44:45.186274 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0925 02:44:45.187137 (+ 863us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:44:45.187948 (+ 811us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:44:45.187957 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:44:45.188716 (+ 759us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:44:45.191711 (+ 2995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:44:45.191735 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:44:45.193249 (+ 1514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:44:45.193253 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:44:45.194321 (+ 1068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:44:45.194332 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:44:45.194957 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:44:45.194961 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:44:45.195152 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:44:45.195880 (+ 728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:44:45.195902 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:44:45.197580 (+ 1678us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:44:45.200482 (+ 2902us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:44:45.200489 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:44:45.200506 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:44:45.200872 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:44:45.201409 (+ 537us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:44:45.201412 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:44:45.201414 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:44:45.201608 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:44:45.202102 (+ 494us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:44:45.202109 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:44:45.202446 (+ 337us) client_negotiation.cc:769] Sending connection context
0925 02:44:45.205195 (+ 2749us) client_negotiation.cc:240] Negotiation successful
0925 02:44:45.205491 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1138,"thread_start_us":127,"threads_started":1}
I20260925 02:44:45.208139 25841 heartbeater.cc:347] Connected to a master server at 127.13.157.190:46205
I20260925 02:44:45.208454 25841 heartbeater.cc:464] Registering TS with master...
I20260925 02:44:45.209086 25841 heartbeater.cc:511] Master 127.13.157.190:46205 requested a full tablet report, sending...
I20260925 02:44:45.210206 25348 ts_manager.cc:194] Registered new tserver with Master: a4aa579542654e54bef80d5e41c47e0a (127.13.157.131:37883)
I20260925 02:44:45.210969 25348 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.157.131@KRBTEST.COM'} at 127.13.157.131:39979
I20260925 02:44:45.215951 13942 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 02:44:45.234443 25569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:44:45.223365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37134 (local address 127.13.157.190:46205)
0925 02:44:45.223488 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0925 02:44:45.223492 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:44:45.226640 (+ 3148us) server_negotiation.cc:407] Connection header received
0925 02:44:45.226808 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:44:45.226811 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:44:45.226863 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:44:45.226947 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:44:45.227822 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:44:45.228338 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:44:45.229468 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:44:45.229696 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:44:45.229799 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:44:45.231438 (+ 1639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:44:45.231442 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:44:45.231445 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:44:45.231472 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:44:45.232854 (+ 1382us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:44:45.233303 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:44:45.233307 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:44:45.233309 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:44:45.233383 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:44:45.233624 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:44:45.233628 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:44:45.233630 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:44:45.233808 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:44:45.233923 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0925 02:44:45.234164 (+ 241us) server_negotiation.cc:299] Negotiation successful
0925 02:44:45.234275 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260925 02:44:45.237685 25348 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37134:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 02:44:45.242327 25348 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 02:44:45.265368 25855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:44:45.258373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.129:36939 (local address 127.0.0.1:49734)
0925 02:44:45.259994 (+ 1621us) negotiation.cc:107] Waiting for socket to connect
0925 02:44:45.260004 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 02:44:45.260070 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:44:45.260714 (+ 644us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:44:45.260717 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:44:45.260736 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:44:45.260820 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:44:45.260825 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:44:45.262179 (+ 1354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:44:45.262182 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:44:45.263559 (+ 1377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:44:45.263570 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:44:45.264811 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:44:45.264814 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:44:45.264925 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:44:45.264963 (+ 38us) client_negotiation.cc:769] Sending connection context
0925 02:44:45.265070 (+ 107us) client_negotiation.cc:240] Negotiation successful
0925 02:44:45.265196 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1569,"mutex_wait_us":35,"thread_start_us":79,"threads_started":1}
I20260925 02:44:45.265615 25853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:44:45.257746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.131:37883 (local address 127.0.0.1:36070)
0925 02:44:45.259002 (+ 1256us) negotiation.cc:107] Waiting for socket to connect
0925 02:44:45.259014 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 02:44:45.259098 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:44:45.261419 (+ 2321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:44:45.261422 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:44:45.261440 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:44:45.261540 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:44:45.261546 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:44:45.263083 (+ 1537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:44:45.263087 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:44:45.264231 (+ 1144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:44:45.264240 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:44:45.265248 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:44:45.265252 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:44:45.265347 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:44:45.265371 (+ 24us) client_negotiation.cc:769] Sending connection context
0925 02:44:45.265417 (+ 46us) client_negotiation.cc:240] Negotiation successful
0925 02:44:45.265481 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1193,"mutex_wait_us":3,"thread_start_us":117,"threads_started":1}
I20260925 02:44:45.266003 25857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:44:45.258604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49734 (local address 127.13.157.129:36939)
0925 02:44:45.260293 (+ 1689us) server_negotiation.cc:206] Beginning negotiation
0925 02:44:45.260300 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 02:44:45.260322 (+ 22us) server_negotiation.cc:407] Connection header received
0925 02:44:45.260391 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:44:45.260395 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:44:45.260583 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:44:45.260720 (+ 137us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:44:45.261019 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:44:45.262051 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:44:45.263696 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:44:45.264489 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:44:45.264608 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:44:45.264744 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0925 02:44:45.265765 (+ 1021us) server_negotiation.cc:299] Negotiation successful
0925 02:44:45.265870 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1587,"thread_start_us":83,"threads_started":1}
I20260925 02:44:45.266505 25858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:44:45.258965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36070 (local address 127.13.157.131:37883)
0925 02:44:45.261013 (+ 2048us) server_negotiation.cc:206] Beginning negotiation
0925 02:44:45.261023 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 02:44:45.261043 (+ 20us) server_negotiation.cc:407] Connection header received
0925 02:44:45.261112 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:44:45.261117 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:44:45.261283 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:44:45.261677 (+ 394us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:44:45.261833 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:44:45.262967 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:44:45.264395 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:44:45.265126 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:44:45.265876 (+ 750us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:44:45.266044 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0925 02:44:45.266279 (+ 235us) server_negotiation.cc:299] Negotiation successful
0925 02:44:45.266374 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1939,"thread_start_us":95,"threads_started":1}
I20260925 02:44:45.267285 25854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:44:45.257616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.130:35321 (local address 127.0.0.1:36192)
0925 02:44:45.258165 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0925 02:44:45.258214 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0925 02:44:45.258344 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:44:45.259823 (+ 1479us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:44:45.259829 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:44:45.259854 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:44:45.259991 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:44:45.259997 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:44:45.261558 (+ 1561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:44:45.261561 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:44:45.263529 (+ 1968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:44:45.263540 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:44:45.264813 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:44:45.264816 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:44:45.266988 (+ 2172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:44:45.267012 (+ 24us) client_negotiation.cc:769] Sending connection context
0925 02:44:45.267082 (+ 70us) client_negotiation.cc:240] Negotiation successful
0925 02:44:45.267148 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":382,"mutex_wait_us":1,"thread_start_us":107,"threads_started":1}
I20260925 02:44:45.267922 25856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:44:45.258450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36192 (local address 127.13.157.130:35321)
0925 02:44:45.259328 (+ 878us) server_negotiation.cc:206] Beginning negotiation
0925 02:44:45.259336 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 02:44:45.259358 (+ 22us) server_negotiation.cc:407] Connection header received
0925 02:44:45.259437 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:44:45.259443 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:44:45.259669 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:44:45.259802 (+ 133us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:44:45.260119 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:44:45.261447 (+ 1328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:44:45.263689 (+ 2242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:44:45.264527 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:44:45.264635 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:44:45.264751 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0925 02:44:45.267684 (+ 2933us) server_negotiation.cc:299] Negotiation successful
0925 02:44:45.267785 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":780,"thread_start_us":114,"threads_started":1}
I20260925 02:44:45.269088 25759 tablet_service.cc:1467] Processing CreateTablet for tablet df8ea11a517f4d3cbaaada89f15a142c (DEFAULT_TABLE table=test-table [id=6e5c39a029a04bf0979ec7e2680e50b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:44:45.269255 25495 tablet_service.cc:1467] Processing CreateTablet for tablet df8ea11a517f4d3cbaaada89f15a142c (DEFAULT_TABLE table=test-table [id=6e5c39a029a04bf0979ec7e2680e50b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:44:45.270210 25759 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df8ea11a517f4d3cbaaada89f15a142c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:44:45.270356 25495 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df8ea11a517f4d3cbaaada89f15a142c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:44:45.273651 25638 tablet_service.cc:1467] Processing CreateTablet for tablet df8ea11a517f4d3cbaaada89f15a142c (DEFAULT_TABLE table=test-table [id=6e5c39a029a04bf0979ec7e2680e50b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:44:45.274757 25638 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df8ea11a517f4d3cbaaada89f15a142c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:44:45.280506 25859 tablet_bootstrap.cc:492] T df8ea11a517f4d3cbaaada89f15a142c P fcf328843180471eb5d2020eb5237979: Bootstrap starting.
I20260925 02:44:45.282318 25861 tablet_bootstrap.cc:492] T df8ea11a517f4d3cbaaada89f15a142c P 8bb07daa6a7d48a1b6eb198cda727f72: Bootstrap starting.
I20260925 02:44:45.282869 25859 tablet_bootstrap.cc:654] T df8ea11a517f4d3cbaaada89f15a142c P fcf328843180471eb5d2020eb5237979: Neither blocks nor log segments found. Creating new log.
I20260925 02:44:45.283252 25860 tablet_bootstrap.cc:492] T df8ea11a517f4d3cbaaada89f15a142c P a4aa579542654e54bef80d5e41c47e0a: Bootstrap starting.
I20260925 02:44:45.283787 25859 log.cc:824] T df8ea11a517f4d3cbaaada89f15a142c P fcf328843180471eb5d2020eb5237979: Log is configured to *not* fsync() on all Append() calls
I20260925 02:44:45.285317 25861 tablet_bootstrap.cc:654] T df8ea11a517f4d3cbaaada89f15a142c P 8bb07daa6a7d48a1b6eb198cda727f72: Neither blocks nor log segments found. Creating new log.
I20260925 02:44:45.286137 25859 tablet_bootstrap.cc:492] T df8ea11a517f4d3cbaaada89f15a142c P fcf328843180471eb5d2020eb5237979: No bootstrap required, opened a new log
I20260925 02:44:45.286342 25860 tablet_bootstrap.cc:654] T df8ea11a517f4d3cbaaada89f15a142c P a4aa579542654e54bef80d5e41c47e0a: Neither blocks nor log segments found. Creating new log.
I20260925 02:44:45.286369 25859 ts_tablet_manager.cc:1397] T df8ea11a517f4d3cbaaada89f15a142c P fcf328843180471eb5d2020eb5237979: Time spent bootstrapping tablet: real 0.006s user 0.006s sys 0.000s
I20260925 02:44:45.286348 25861 log.cc:824] T df8ea11a517f4d3cbaaada89f15a142c P 8bb07daa6a7d48a1b6eb198cda727f72: Log is configured to *not* fsync() on all Append() calls
I20260925 02:44:45.287264 25860 log.cc:824] T df8ea11a517f4d3cbaaada89f15a142c P a4aa579542654e54bef80d5e41c47e0a: Log is configured to *not* fsync() on all Append() calls
I20260925 02:44:45.288600 25861 tablet_bootstrap.cc:492] T df8ea11a517f4d3cbaaada89f15a142c P 8bb07daa6a7d48a1b6eb198cda727f72: No bootstrap required, opened a new log
I20260925 02:44:45.288803 25861 ts_tablet_manager.cc:1397] T df8ea11a517f4d3cbaaada89f15a142c P 8bb07daa6a7d48a1b6eb198cda727f72: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260925 02:44:45.290478 25859 raft_consensus.cc:348] T df8ea11a517f4d3cbaaada89f15a142c P fcf328843180471eb5d2020eb5237979 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bb07daa6a7d48a1b6eb198cda727f72" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 35321 } } peers { permanent_uuid: "a4aa579542654e54bef80d5e41c47e0a" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 37883 } } peers { permanent_uuid: "fcf328843180471eb5d2020eb5237979" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 36939 } }
I20260925 02:44:45.290769 25859 raft_consensus.cc:374] T df8ea11a517f4d3cbaaada89f15a142c P fcf328843180471eb5d2020eb5237979 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:44:45.290818 25859 raft_consensus.cc:729] T df8ea11a517f4d3cbaaada89f15a142c P fcf328843180471eb5d2020eb5237979 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcf328843180471eb5d2020eb5237979, State: Initialized, Role: FOLLOWER
I20260925 02:44:45.291443 25860 tablet_bootstrap.cc:492] T df8ea11a517f4d3cbaaada89f15a142c P a4aa579542654e54bef80d5e41c47e0a: No bootstrap required, opened a new log
I20260925 02:44:45.291612 25860 ts_tablet_manager.cc:1397] T df8ea11a517f4d3cbaaada89f15a142c P a4aa579542654e54bef80d5e41c47e0a: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.000s
I20260925 02:44:45.292455 25859 consensus_queue.cc:260] T df8ea11a517f4d3cbaaada89f15a142c P fcf328843180471eb5d2020eb5237979 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bb07daa6a7d48a1b6eb198cda727f72" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 35321 } } peers { permanent_uuid: "a4aa579542654e54bef80d5e41c47e0a" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 37883 } } peers { permanent_uuid: "fcf328843180471eb5d2020eb5237979" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 36939 } }
I20260925 02:44:45.292479 25861 raft_consensus.cc:348] T df8ea11a517f4d3cbaaada89f15a142c P 8bb07daa6a7d48a1b6eb198cda727f72 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bb07daa6a7d48a1b6eb198cda727f72" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 35321 } } peers { permanent_uuid: "a4aa579542654e54bef80d5e41c47e0a" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 37883 } } peers { permanent_uuid: "fcf328843180471eb5d2020eb5237979" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 36939 } }
I20260925 02:44:45.292714 25861 raft_consensus.cc:374] T df8ea11a517f4d3cbaaada89f15a142c P 8bb07daa6a7d48a1b6eb198cda727f72 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:44:45.292773 25861 raft_consensus.cc:729] T df8ea11a517f4d3cbaaada89f15a142c P 8bb07daa6a7d48a1b6eb198cda727f72 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8bb07daa6a7d48a1b6eb198cda727f72, State: Initialized, Role: FOLLOWER
I20260925 02:44:45.293190 25861 consensus_queue.cc:260] T df8ea11a517f4d3cbaaada89f15a142c P 8bb07daa6a7d48a1b6eb198cda727f72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bb07daa6a7d48a1b6eb198cda727f72" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 35321 } } peers { permanent_uuid: "a4aa579542654e54bef80d5e41c47e0a" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 37883 } } peers { permanent_uuid: "fcf328843180471eb5d2020eb5237979" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 36939 } }
I20260925 02:44:45.293582 25859 ts_tablet_manager.cc:1428] T df8ea11a517f4d3cbaaada89f15a142c P fcf328843180471eb5d2020eb5237979: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20260925 02:44:45.293957 25566 heartbeater.cc:503] Master 127.13.157.190:46205 was elected leader, sending a full tablet report...
I20260925 02:44:45.294135 25861 ts_tablet_manager.cc:1428] T df8ea11a517f4d3cbaaada89f15a142c P 8bb07daa6a7d48a1b6eb198cda727f72: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260925 02:44:45.295378 25860 raft_consensus.cc:348] T df8ea11a517f4d3cbaaada89f15a142c P a4aa579542654e54bef80d5e41c47e0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bb07daa6a7d48a1b6eb198cda727f72" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 35321 } } peers { permanent_uuid: "a4aa579542654e54bef80d5e41c47e0a" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 37883 } } peers { permanent_uuid: "fcf328843180471eb5d2020eb5237979" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 36939 } }
I20260925 02:44:45.295706 25860 raft_consensus.cc:374] T df8ea11a517f4d3cbaaada89f15a142c P a4aa579542654e54bef80d5e41c47e0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:44:45.295794 25860 raft_consensus.cc:729] T df8ea11a517f4d3cbaaada89f15a142c P a4aa579542654e54bef80d5e41c47e0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4aa579542654e54bef80d5e41c47e0a, State: Initialized, Role: FOLLOWER
I20260925 02:44:45.296280 25860 consensus_queue.cc:260] T df8ea11a517f4d3cbaaada89f15a142c P a4aa579542654e54bef80d5e41c47e0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bb07daa6a7d48a1b6eb198cda727f72" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 35321 } } peers { permanent_uuid: "a4aa579542654e54bef80d5e41c47e0a" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 37883 } } peers { permanent_uuid: "fcf328843180471eb5d2020eb5237979" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 36939 } }
I20260925 02:44:45.297205 25860 ts_tablet_manager.cc:1428] T df8ea11a517f4d3cbaaada89f15a142c P a4aa579542654e54bef80d5e41c47e0a: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260925 02:44:45.297585 25841 heartbeater.cc:503] Master 127.13.157.190:46205 was elected leader, sending a full tablet report...
I20260925 02:44:45.299189 25705 heartbeater.cc:503] Master 127.13.157.190:46205 was elected leader, sending a full tablet report...
W20260925 02:44:45.331955 25706 tablet.cc:2369] T df8ea11a517f4d3cbaaada89f15a142c P 8bb07daa6a7d48a1b6eb198cda727f72: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 02:44:45.389930 25865 raft_consensus.cc:482] T df8ea11a517f4d3cbaaada89f15a142c P fcf328843180471eb5d2020eb5237979 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 02:44:45.390183 25865 raft_consensus.cc:504] T df8ea11a517f4d3cbaaada89f15a142c P fcf328843180471eb5d2020eb5237979 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bb07daa6a7d48a1b6eb198cda727f72" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 35321 } } peers { permanent_uuid: "a4aa579542654e54bef80d5e41c47e0a" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 37883 } } peers { permanent_uuid: "fcf328843180471eb5d2020eb5237979" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 36939 } }
I20260925 02:44:45.391671 25865 leader_election.cc:288] T df8ea11a517f4d3cbaaada89f15a142c P fcf328843180471eb5d2020eb5237979 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8bb07daa6a7d48a1b6eb198cda727f72 (127.13.157.130:35321), a4aa579542654e54bef80d5e41c47e0a (127.13.157.131:37883)
I20260925 02:44:45.397454 25856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:44:45.392401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.129:44649 (local address 127.13.157.130:35321)
0925 02:44:45.392530 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0925 02:44:45.392534 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:44:45.392780 (+ 246us) server_negotiation.cc:407] Connection header received
0925 02:44:45.392984 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:44:45.392988 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:44:45.393037 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:44:45.393083 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:44:45.393547 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:44:45.394359 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:44:45.396075 (+ 1716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:44:45.396701 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:44:45.396799 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:44:45.396856 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0925 02:44:45.397219 (+ 363us) server_negotiation.cc:299] Negotiation successful
0925 02:44:45.397314 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260925 02:44:45.398098 25869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:44:45.391924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.130:35321 (local address 127.13.157.129:44649)
0925 02:44:45.392685 (+ 761us) negotiation.cc:107] Waiting for socket to connect
0925 02:44:45.392700 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 02:44:45.392863 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:44:45.393238 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:44:45.393241 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:44:45.393276 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:44:45.393419 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:44:45.393425 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:44:45.394504 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:44:45.394507 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:44:45.395939 (+ 1432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:44:45.395950 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:44:45.396966 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:44:45.396970 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:44:45.397072 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:44:45.397097 (+ 25us) client_negotiation.cc:769] Sending connection context
0925 02:44:45.397850 (+ 753us) client_negotiation.cc:240] Negotiation successful
0925 02:44:45.397932 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":694,"thread_start_us":146,"threads_started":1}
I20260925 02:44:45.398973 25659 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df8ea11a517f4d3cbaaada89f15a142c" candidate_uuid: "fcf328843180471eb5d2020eb5237979" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bb07daa6a7d48a1b6eb198cda727f72" is_pre_election: true
I20260925 02:44:45.399333 25659 raft_consensus.cc:2454] T df8ea11a517f4d3cbaaada89f15a142c P 8bb07daa6a7d48a1b6eb198cda727f72 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf328843180471eb5d2020eb5237979 in term 0.
I20260925 02:44:45.399996 25451 leader_election.cc:302] T df8ea11a517f4d3cbaaada89f15a142c P fcf328843180471eb5d2020eb5237979 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8bb07daa6a7d48a1b6eb198cda727f72, fcf328843180471eb5d2020eb5237979; no voters:
I20260925 02:44:45.400321 25865 raft_consensus.cc:2781] T df8ea11a517f4d3cbaaada89f15a142c P fcf328843180471eb5d2020eb5237979 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 02:44:45.400396 25865 raft_consensus.cc:482] T df8ea11a517f4d3cbaaada89f15a142c P fcf328843180471eb5d2020eb5237979 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 02:44:45.400436 25865 raft_consensus.cc:3037] T df8ea11a517f4d3cbaaada89f15a142c P fcf328843180471eb5d2020eb5237979 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:44:45.401731 25865 raft_consensus.cc:504] T df8ea11a517f4d3cbaaada89f15a142c P fcf328843180471eb5d2020eb5237979 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bb07daa6a7d48a1b6eb198cda727f72" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 35321 } } peers { permanent_uuid: "a4aa579542654e54bef80d5e41c47e0a" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 37883 } } peers { permanent_uuid: "fcf328843180471eb5d2020eb5237979" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 36939 } }
I20260925 02:44:45.402104 25865 leader_election.cc:288] T df8ea11a517f4d3cbaaada89f15a142c P fcf328843180471eb5d2020eb5237979 [CANDIDATE]: Term 1 election: Requested vote from peers 8bb07daa6a7d48a1b6eb198cda727f72 (127.13.157.130:35321), a4aa579542654e54bef80d5e41c47e0a (127.13.157.131:37883)
I20260925 02:44:45.402701 25659 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df8ea11a517f4d3cbaaada89f15a142c" candidate_uuid: "fcf328843180471eb5d2020eb5237979" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bb07daa6a7d48a1b6eb198cda727f72"
I20260925 02:44:45.402822 25659 raft_consensus.cc:3037] T df8ea11a517f4d3cbaaada89f15a142c P 8bb07daa6a7d48a1b6eb198cda727f72 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:44:45.404106 25659 raft_consensus.cc:2454] T df8ea11a517f4d3cbaaada89f15a142c P 8bb07daa6a7d48a1b6eb198cda727f72 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf328843180471eb5d2020eb5237979 in term 1.
I20260925 02:44:45.404531 25451 leader_election.cc:302] T df8ea11a517f4d3cbaaada89f15a142c P fcf328843180471eb5d2020eb5237979 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8bb07daa6a7d48a1b6eb198cda727f72, fcf328843180471eb5d2020eb5237979; no voters:
I20260925 02:44:45.404757 25865 raft_consensus.cc:2781] T df8ea11a517f4d3cbaaada89f15a142c P fcf328843180471eb5d2020eb5237979 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:44:45.404983 25865 raft_consensus.cc:686] T df8ea11a517f4d3cbaaada89f15a142c P fcf328843180471eb5d2020eb5237979 [term 1 LEADER]: Becoming Leader. State: Replica: fcf328843180471eb5d2020eb5237979, State: Running, Role: LEADER
I20260925 02:44:45.405318 25858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:44:45.392403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.129:55905 (local address 127.13.157.131:37883)
0925 02:44:45.392530 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0925 02:44:45.392534 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:44:45.393954 (+ 1420us) server_negotiation.cc:407] Connection header received
0925 02:44:45.396891 (+ 2937us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:44:45.396894 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:44:45.396950 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:44:45.397000 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:44:45.397426 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:44:45.398132 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:44:45.404120 (+ 5988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:44:45.404717 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:44:45.404858 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:44:45.404912 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0925 02:44:45.405092 (+ 180us) server_negotiation.cc:299] Negotiation successful
0925 02:44:45.405176 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260925 02:44:45.405380 25865 consensus_queue.cc:237] T df8ea11a517f4d3cbaaada89f15a142c P fcf328843180471eb5d2020eb5237979 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bb07daa6a7d48a1b6eb198cda727f72" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 35321 } } peers { permanent_uuid: "a4aa579542654e54bef80d5e41c47e0a" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 37883 } } peers { permanent_uuid: "fcf328843180471eb5d2020eb5237979" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 36939 } }
I20260925 02:44:45.405735 25868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:44:45.392008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.131:37883 (local address 127.13.157.129:55905)
0925 02:44:45.393860 (+ 1852us) negotiation.cc:107] Waiting for socket to connect
0925 02:44:45.393874 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 02:44:45.396739 (+ 2865us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:44:45.397145 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:44:45.397148 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:44:45.397166 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:44:45.397297 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:44:45.397305 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:44:45.403041 (+ 5736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:44:45.403045 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:44:45.403974 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:44:45.403984 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:44:45.404847 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:44:45.404850 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:44:45.404948 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:44:45.404973 (+ 25us) client_negotiation.cc:769] Sending connection context
0925 02:44:45.405511 (+ 538us) client_negotiation.cc:240] Negotiation successful
0925 02:44:45.405593 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1784,"thread_start_us":93,"threads_started":1}
I20260925 02:44:45.406562 25781 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df8ea11a517f4d3cbaaada89f15a142c" candidate_uuid: "fcf328843180471eb5d2020eb5237979" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4aa579542654e54bef80d5e41c47e0a" is_pre_election: true
I20260925 02:44:45.406849 25781 raft_consensus.cc:2454] T df8ea11a517f4d3cbaaada89f15a142c P a4aa579542654e54bef80d5e41c47e0a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf328843180471eb5d2020eb5237979 in term 0.
I20260925 02:44:45.407210 25780 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df8ea11a517f4d3cbaaada89f15a142c" candidate_uuid: "fcf328843180471eb5d2020eb5237979" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4aa579542654e54bef80d5e41c47e0a"
I20260925 02:44:45.407333 25780 raft_consensus.cc:3037] T df8ea11a517f4d3cbaaada89f15a142c P a4aa579542654e54bef80d5e41c47e0a [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:44:45.408659 25780 raft_consensus.cc:2454] T df8ea11a517f4d3cbaaada89f15a142c P a4aa579542654e54bef80d5e41c47e0a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf328843180471eb5d2020eb5237979 in term 1.
I20260925 02:44:45.409106 25348 catalog_manager.cc:5495] T df8ea11a517f4d3cbaaada89f15a142c P fcf328843180471eb5d2020eb5237979 reported cstate change: term changed from 0 to 1, leader changed from <none> to fcf328843180471eb5d2020eb5237979 (127.13.157.129). New cstate: current_term: 1 leader_uuid: "fcf328843180471eb5d2020eb5237979" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bb07daa6a7d48a1b6eb198cda727f72" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 35321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4aa579542654e54bef80d5e41c47e0a" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 37883 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf328843180471eb5d2020eb5237979" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 36939 } health_report { overall_health: HEALTHY } } }
I20260925 02:44:45.423591 25857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:44:45.418872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49736 (local address 127.13.157.129:36939)
0925 02:44:45.419046 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0925 02:44:45.419051 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:44:45.419066 (+ 15us) server_negotiation.cc:407] Connection header received
0925 02:44:45.419143 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:44:45.419146 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:44:45.419208 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:44:45.419323 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 02:44:45.419621 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:44:45.420426 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:44:45.421781 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:44:45.422119 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:44:45.422214 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:44:45.422643 (+ 429us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 02:44:45.423188 (+ 545us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 02:44:45.423312 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0925 02:44:45.423392 (+ 80us) server_negotiation.cc:299] Negotiation successful
0925 02:44:45.423453 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260925 02:44:45.430639 25780 raft_consensus.cc:1261] T df8ea11a517f4d3cbaaada89f15a142c P a4aa579542654e54bef80d5e41c47e0a [term 1 FOLLOWER]: Refusing update from remote peer fcf328843180471eb5d2020eb5237979: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:44:45.431411 25659 raft_consensus.cc:1261] T df8ea11a517f4d3cbaaada89f15a142c P 8bb07daa6a7d48a1b6eb198cda727f72 [term 1 FOLLOWER]: Refusing update from remote peer fcf328843180471eb5d2020eb5237979: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:44:45.431844 25870 consensus_queue.cc:1035] T df8ea11a517f4d3cbaaada89f15a142c P fcf328843180471eb5d2020eb5237979 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4aa579542654e54bef80d5e41c47e0a" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 37883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:44:45.432034 25865 consensus_queue.cc:1035] T df8ea11a517f4d3cbaaada89f15a142c P fcf328843180471eb5d2020eb5237979 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bb07daa6a7d48a1b6eb198cda727f72" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 35321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260925 02:44:45.435952 25842 tablet.cc:2369] T df8ea11a517f4d3cbaaada89f15a142c P a4aa579542654e54bef80d5e41c47e0a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 02:44:45.439565 25874 mvcc.cc:205] Tried to move back new op lower bound from 7333086353115717632 to 7333086353023787008. Current Snapshot: MvccSnapshot[applied={T|T < 7333086353115717632}]
Sep 25 02:44:45 dist-test-slave-kznw krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304284, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.190@KRBTEST.COM for HTTP/127.13.157.148@KRBTEST.COM
I20260925 02:44:45.442814 25875 mvcc.cc:205] Tried to move back new op lower bound from 7333086353115717632 to 7333086353023787008. Current Snapshot: MvccSnapshot[applied={T|T < 7333086353115717632}]
I20260925 02:44:45.454572 25876 mvcc.cc:205] Tried to move back new op lower bound from 7333086353115717632 to 7333086353023787008. Current Snapshot: MvccSnapshot[applied={T|T < 7333086353115717632 or (T in {7333086353115717632})}]
I20260925 02:44:45.455785 25348 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37134:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 02:44:45.455973 25348 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37134:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 02:44:45.458616 25348 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cd798d439007493a9e7bd4fa5dbef9f7: Sending DeleteTablet for 3 replicas of tablet df8ea11a517f4d3cbaaada89f15a142c
I20260925 02:44:45.459393 25638 tablet_service.cc:1514] Processing DeleteTablet for tablet df8ea11a517f4d3cbaaada89f15a142c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:44:45 UTC) from {username='kudu', principal='kudu/127.13.157.190@KRBTEST.COM'} at 127.0.0.1:36192
I20260925 02:44:45.459650 25759 tablet_service.cc:1514] Processing DeleteTablet for tablet df8ea11a517f4d3cbaaada89f15a142c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:44:45 UTC) from {username='kudu', principal='kudu/127.13.157.190@KRBTEST.COM'} at 127.0.0.1:36070
I20260925 02:44:45.460112 25495 tablet_service.cc:1514] Processing DeleteTablet for tablet df8ea11a517f4d3cbaaada89f15a142c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:44:45 UTC) from {username='kudu', principal='kudu/127.13.157.190@KRBTEST.COM'} at 127.0.0.1:49734
I20260925 02:44:45.460666 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 25431
I20260925 02:44:45.462298 25884 tablet_replica.cc:330] T df8ea11a517f4d3cbaaada89f15a142c P a4aa579542654e54bef80d5e41c47e0a: stopping tablet replica
I20260925 02:44:45.469369 25884 raft_consensus.cc:2229] T df8ea11a517f4d3cbaaada89f15a142c P a4aa579542654e54bef80d5e41c47e0a [term 1 FOLLOWER]: Raft consensus shutting down.
W20260925 02:44:45.470450 25332 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.157.129:36939 (error 108)
I20260925 02:44:45.470748 25884 raft_consensus.cc:2258] T df8ea11a517f4d3cbaaada89f15a142c P a4aa579542654e54bef80d5e41c47e0a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:44:45.471271 25883 tablet_replica.cc:330] T df8ea11a517f4d3cbaaada89f15a142c P 8bb07daa6a7d48a1b6eb198cda727f72: stopping tablet replica
I20260925 02:44:45.471457 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 25574
I20260925 02:44:45.472687 25854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:44:45.472375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.129:36939 (local address 127.0.0.1:49744)
0925 02:44:45.472514 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0925 02:44:45.472596 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.129:36939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260925 02:44:45.472909 25332 catalog_manager.cc:4584] TS fcf328843180471eb5d2020eb5237979 (127.13.157.129:36939): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet df8ea11a517f4d3cbaaada89f15a142c: Network error: Client connection negotiation failed: client connection to 127.13.157.129:36939: connect: Connection refused (error 111)
I20260925 02:44:45.477417 25854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:44:45.477159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.130:35321 (local address 127.0.0.1:36196)
0925 02:44:45.477277 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0925 02:44:45.477335 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.130:35321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260925 02:44:45.478502 25884 ts_tablet_manager.cc:1905] T df8ea11a517f4d3cbaaada89f15a142c P a4aa579542654e54bef80d5e41c47e0a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:44:45.481237 25884 ts_tablet_manager.cc:1918] T df8ea11a517f4d3cbaaada89f15a142c P a4aa579542654e54bef80d5e41c47e0a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 02:44:45.481335 25884 log.cc:1196] T df8ea11a517f4d3cbaaada89f15a142c P a4aa579542654e54bef80d5e41c47e0a: Deleting WAL directory at /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790304178325258-13942-0/minicluster-data/ts-2/wal/wals/df8ea11a517f4d3cbaaada89f15a142c
I20260925 02:44:45.481537 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 25710
I20260925 02:44:45.487453 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 25316
I20260925 02:44:45.487691 25854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:44:45.487411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.131:37883 (local address 127.0.0.1:36074)
0925 02:44:45.487550 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0925 02:44:45.487608 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.131:37883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260925 02:44:45.495941 13942 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260925 02:44:45.840296 13942 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260925 02:44:45.840441 13942 mini_ranger.cc:66] Stopping Ranger...
I20260925 02:44:46.174702 13942 mini_ranger.cc:68] Stopped Ranger
2026-09-25 02:44:46.174 UTC [24899] LOG: received smart shutdown request
2026-09-25 02:44:46.178 UTC [24899] LOG: background worker "logical replication launcher" (PID 24908) exited with exit code 1
2026-09-25 02:44:46.178 UTC [24903] LOG: shutting down
2026-09-25 02:44:46.344 UTC [24899] LOG: database system is shut down
2026-09-25T02:44:46Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (45804 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 02:44:46 dist-test-slave-kznw krb5kdc[25896](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 02:44:46 dist-test-slave-kznw krb5kdc[25896](info): set up 2 sockets
Sep 25 02:44:46 dist-test-slave-kznw krb5kdc[25896](info): commencing operation
krb5kdc: starting...
W20260925 02:44:47.398522 13942 mini_kdc.cc:121] Time spent starting KDC: real 0.938s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/krb5kdc/test-user.keytab.
Sep 25 02:44:47 dist-test-slave-kznw krb5kdc[25896](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304287, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T02:44:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T02:44:47Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.13.157.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.13.157.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.13.157.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/krb5kdc/rangeradmin_127.13.157.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.13.157.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/krb5kdc/rangeradmin_127.13.157.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.13.157.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.13.157.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.13.157.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/krb5kdc/rangerlookup_127.13.157.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.13.157.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/krb5kdc/rangerlookup_127.13.157.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.13.157.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.13.157.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.13.157.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/krb5kdc/HTTP_127.13.157.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.13.157.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/krb5kdc/HTTP_127.13.157.148@KRBTEST.COM.keytab.
I20260925 02:44:47.553435 13942 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-tasksiQ9fF/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/postgres -l logfile start
2026-09-25 02:44:49.924 UTC [25926] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-25 02:44:49.924 UTC [25926] LOG: listening on IPv4 address "127.13.157.148", port 48303
2026-09-25 02:44:49.927 UTC [25926] LOG: listening on Unix socket "/tmp/.s.PGSQL.48303"
2026-09-25 02:44:49.948 UTC [25929] LOG: database system was shut down at 2026-09-25 02:44:48 UTC
2026-09-25 02:44:49.951 UTC [25926] LOG: database system is ready to accept connections
I20260925 02:44:50.818269 13942 mini_postgres.cc:96] Postgres bound to 48303
2026-09-25 02:44:50.824 UTC [25937] FATAL: database "slave" does not exist
127.13.157.148:48303 - accepting connections
I20260925 02:44:50.824826 13942 mini_ranger.cc:161] Starting Ranger...
I20260925 02:44:50.839221 13942 mini_ranger.cc:84] Created miniranger Postgres user
I20260925 02:44:51.305791 13942 mini_ranger.cc:87] Created ranger Postgres database
I20260925 02:44:51.305872 13942 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/ranger-admin
2026-09-25 02:44:51,573 [I] DB FLAVOR :POSTGRES
2026-09-25 02:44:51,573 [I] --------- Verifying Ranger DB connection ---------
2026-09-25 02:44:51,573 [I] Checking connection..
2026-09-25 02:44:51,573 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksiQ9fF/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksiQ9fF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.157.148:48303/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-25 02:44:51,823 [I] Checking connection passed.
2026-09-25 02:44:51,824 [I] --------- Verifying version history table ---------
2026-09-25 02:44:51,824 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksiQ9fF/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksiQ9fF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.157.148:48303/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-25 02:44:52,063 [I] Table x_db_version_h does not exist in database ranger
2026-09-25 02:44:52,063 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksiQ9fF/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksiQ9fF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.157.148:48303/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-25 02:44:52,306 [I] Table x_db_version_h does not exist in database ranger
2026-09-25 02:44:52,306 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-25 02:44:52,306 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksiQ9fF/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksiQ9fF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.157.148:48303/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasksiQ9fF/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-25 02:44:52.544 UTC [26001] WARNING: there is no transaction in progress
2026-09-25 02:44:52,550 [I] create_dbversion_catalog.sql file imported successfully
2026-09-25 02:44:52,550 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksiQ9fF/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksiQ9fF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.157.148:48303/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-25 02:44:52,807 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-25 02:44:52,807 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-25 02:44:52,807 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksiQ9fF/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksiQ9fF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.157.148:48303/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-25 02:44:53,063 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksiQ9fF/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksiQ9fF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.157.148:48303/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-25 02:44:53,294 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksiQ9fF/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksiQ9fF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.157.148:48303/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-kznw.c.gcp-upstream.internal','N') ;"
2026-09-25 02:44:53,532 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksiQ9fF/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksiQ9fF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.157.148:48303/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-25 02:44:53,770 [I] Table x_portal_user does not exist in database ranger
2026-09-25 02:44:53,771 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksiQ9fF/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksiQ9fF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.157.148:48303/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-25 02:44:53,994 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-25 02:44:53,995 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksiQ9fF/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksiQ9fF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.157.148:48303/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-25 02:44:54,253 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksiQ9fF/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksiQ9fF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.157.148:48303/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-25 02:44:54,508 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-25 02:44:54,508 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksiQ9fF/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksiQ9fF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.157.148:48303/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasksiQ9fF/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-25 02:44:55.422 UTC [26112] WARNING: there is no transaction in progress
2026-09-25 02:44:55.438 UTC [26112] WARNING: there is no transaction in progress
2026-09-25 02:44:55.454 UTC [26112] WARNING: there is no transaction in progress
2026-09-25 02:44:55.469 UTC [26112] WARNING: there is no transaction in progress
2026-09-25 02:44:55.484 UTC [26112] WARNING: there is no transaction in progress
2026-09-25 02:44:55.502 UTC [26112] WARNING: there is no transaction in progress
2026-09-25 02:44:55.559 UTC [26112] WARNING: there is no transaction in progress
2026-09-25 02:44:55.565 UTC [26112] WARNING: there is no transaction in progress
2026-09-25 02:44:55.580 UTC [26112] WARNING: there is no transaction in progress
2026-09-25 02:44:55.589 UTC [26112] WARNING: there is no transaction in progress
2026-09-25 02:44:55.599 UTC [26112] WARNING: there is no transaction in progress
2026-09-25 02:44:55.613 UTC [26112] WARNING: there is no transaction in progress
2026-09-25 02:44:55.623 UTC [26112] WARNING: there is no transaction in progress
2026-09-25 02:44:55.630 UTC [26112] WARNING: there is no transaction in progress
2026-09-25 02:44:55.638 UTC [26112] WARNING: there is no transaction in progress
2026-09-25 02:44:55.645 UTC [26112] WARNING: there is no transaction in progress
2026-09-25 02:44:56,273 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-25 02:44:56,273 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksiQ9fF/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksiQ9fF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.157.148:48303/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-25 02:44:56,498 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-25 02:44:56,499 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksiQ9fF/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksiQ9fF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.157.148:48303/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-25 02:44:56,753 [I] Table x_portal_user already exists in database 'ranger'
2026-09-25 02:44:56,754 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksiQ9fF/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksiQ9fF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.157.148:48303/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-25 02:44:56,992 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-25 02:44:56,993 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksiQ9fF/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksiQ9fF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.157.148:48303/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-25 02:44:57,232 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksiQ9fF/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksiQ9fF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.157.148:48303/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-25 02:44:57,479 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksiQ9fF/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksiQ9fF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.157.148:48303/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-kznw.c.gcp-upstream.internal';"
2026-09-25 02:44:57,723 [I] CORE_DB_SCHEMA import status has been updated
2026-09-25 02:44:57,724 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksiQ9fF/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksiQ9fF/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.157.148:48303/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-25 02:44:57,975 [I] DB_PATCHES have already been applied
I20260925 02:44:57.981146 13942 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/ranger-admin:/tmp/dist-test-tasksiQ9fF/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-tasksiQ9fF/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-tasksiQ9fF/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksiQ9fF/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260925 02:44:57.981211 13942 mini_ranger.cc:193] Using host: 127.13.157.148
I20260925 02:44:57.984525 13942 mini_ranger.cc:227] Ranger admin URL: http://127.13.157.148:58235
Sep 25, 2026 2:44:58 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 25, 2026 2:44:58 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 25, 2026 2:44:58 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/ranger-admin/ews/webapp
Sep 25, 2026 2:44:58 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 25, 2026 2:44:58 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 25, 2026 2:44:58 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/ranger-admin/ews/webapp].
Sep 25, 2026 2:44:58 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260925 02:44:58.993884 13942 mini_ranger.cc:160] Time spent starting Ranger: real 8.169s user 0.000s sys 0.009s
Sep 25, 2026 2:45:18 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 25, 2026 2:45:18 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Sep 25, 2026 2:45:18 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 25, 2026 2:45:18 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 25, 2026 2:45:18 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 25, 2026 2:45:18 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 25, 2026 2:45:18 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 25, 2026 2:45:18 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 25, 2026 2:45:18 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 25, 2026 2:45:18 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 25, 2026 2:45:18 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 25, 2026 2:45:18 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 25, 2026 2:45:18 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 25, 2026 2:45:18 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 25, 2026 2:45:18 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 25, 2026 2:45:18 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 25, 2026 2:45:18 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 25, 2026 2:45:19 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260925 02:45:19.622450 13942 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.13.157.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.13.157.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.13.157.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/krb5kdc/rangerkms_127.13.157.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.13.157.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/krb5kdc/rangerkms_127.13.157.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.13.157.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.13.157.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.13.157.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/krb5kdc/HTTP_127.13.157.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.13.157.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/krb5kdc/HTTP_127.13.157.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 25 02:45:19 dist-test-slave-kznw krb5kdc[25896](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304319, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260925 02:45:19.705991 13942 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260925 02:45:19.715306 13942 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260925 02:45:20.584846 13942 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260925 02:45:20.584928 13942 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/ranger-kms
I20260925 02:45:20.584950 13942 mini_ranger_kms.cc:227] Using postgres at 127.13.157.148:48303
2026-09-25 02:45:20,753 [I] DB FLAVOR :POSTGRES
2026-09-25 02:45:20,754 [I] --------- Verifying Ranger DB connection ---------
2026-09-25 02:45:20,754 [I] Checking connection
2026-09-25 02:45:20,754 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksiQ9fF/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.157.148:48303/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-25 02:45:20,996 [I] connection success
2026-09-25 02:45:20,997 [I] --------- Verifying Ranger DB tables ---------
2026-09-25 02:45:20,997 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-25 02:45:20,997 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksiQ9fF/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.157.148:48303/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-25 02:45:21,261 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-25 02:45:21,261 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-25 02:45:21,261 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-25 02:45:21,261 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksiQ9fF/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.157.148:48303/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-25 02:45:21,510 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260925 02:45:21.977789 13942 mini_ranger_kms.cc:310] Created kms service
I20260925 02:45:22.167665 13942 mini_ranger_kms.cc:326] Created kudu user
I20260925 02:45:22.211287 13942 mini_ranger_kms.cc:343] Created rangerkms user
Sep 25 02:45:22 dist-test-slave-kznw krb5kdc[25896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304319, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.13.157.148@KRBTEST.COM
I20260925 02:45:22.579128 13942 mini_ranger_kms.cc:384] Added ranger policy
I20260925 02:45:22.579290 13942 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/ranger-kms:/tmp/dist-test-tasksiQ9fF/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksiQ9fF/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-tasksiQ9fF/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260925 02:45:22.579317 13942 mini_ranger_kms.cc:240] Using host: 127.13.157.148
I20260925 02:45:22.582784 13942 mini_ranger_kms.cc:276] Ranger KMS PID: 26294
I20260925 02:45:22.582870 13942 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.13.157.148:42843
Sep 25, 2026 2:45:23 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 25, 2026 2:45:23 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 25, 2026 2:45:23 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 25, 2026 2:45:23 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 25, 2026 2:45:23 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/ranger-kms/ews/webapp].
Sep 25, 2026 2:45:23 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260925 02:45:24.550884 13942 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 4.845s user 0.023s sys 0.130s
I20260925 02:45:24.551026 13942 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260925 02:45:24.551105 13942 mini_ranger_kms.cc:401] 127.13.157.148:42843/kms/v1/keys
Sep 25 02:45:24 dist-test-slave-kznw krb5kdc[25896](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790304324, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.13.157.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 25 02:45:25 dist-test-slave-kznw krb5kdc[25896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304324, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.13.157.148@KRBTEST.COM for HTTP/127.13.157.148@KRBTEST.COM
Sep 25 02:45:25 dist-test-slave-kznw krb5kdc[25896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304324, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.13.157.148@KRBTEST.COM for HTTP/127.13.157.148@KRBTEST.COM
Sep 25 02:45:27 dist-test-slave-kznw krb5kdc[25896](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304327, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.13.157.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.157.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.157.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab.
Entry for principal kudu/127.13.157.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab.
I20260925 02:45:28.032351 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.157.190:40793
--webserver_interface=127.13.157.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab
--principal=kudu/127.13.157.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:35221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.157.190:40793
--ranger_config_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.157.148:42843/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:45:28.145278 26345 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:45:28.145553 26345 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:45:28.145592 26345 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:45:28.149684 26345 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:45:28.149747 26345 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:45:28.149771 26345 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:45:28.149786 26345 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:45:28.149803 26345 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:45:28.155464 26345 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:35221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/master-0/wal
--ranger_kms_url=127.13.157.148:42843/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.13.157.190:40793
--ranger_config_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab
--principal=kudu/127.13.157.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.157.190:40793
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.157.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26345
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:45:28.156555 26345 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:45:28.157598 26345 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:45:28.167539 26350 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:45:28.167548 26351 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:45:28.167667 26345 server_base.cc:1034] running on GCE node
W20260925 02:45:28.167834 26353 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:45:28.168407 26345 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:45:28.169538 26345 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:45:28.170718 26345 hybrid_clock.cc:648] HybridClock initialized: now 1790304328170696 us; error 38 us; skew 500 ppm
Sep 25 02:45:28 dist-test-slave-kznw krb5kdc[25896](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304328, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:45:28.176960 26345 init.cc:377] Logged in from keytab as kudu/127.13.157.190@KRBTEST.COM (short username kudu)
I20260925 02:45:28.177903 26345 webserver.cc:460] Webserver started at http://127.13.157.190:42935/ using document root <none> and password file <none>
I20260925 02:45:28.178444 26345 fs_manager.cc:359] Metadata directory not provided
I20260925 02:45:28.178498 26345 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:45:28.178689 26345 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 25 02:45:28 dist-test-slave-kznw krb5kdc[25896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304328, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.190@KRBTEST.COM for HTTP/127.13.157.148@KRBTEST.COM
I20260925 02:45:28.225960 26345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/master-0/data/instance:
uuid: "28497343617d4cf396f60d556f7be937"
format_stamp: "Formatted at 2026-09-25 02:45:28 on dist-test-slave-kznw"
server_key: "053658eb96f4939a2d9ccb1c4f1954a8"
server_key_iv: "6d58b042de161a0de5b789cae50f81cf"
server_key_version: "kuduclusterkey@0"
I20260925 02:45:28.226555 26345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/master-0/wal/instance:
uuid: "28497343617d4cf396f60d556f7be937"
format_stamp: "Formatted at 2026-09-25 02:45:28 on dist-test-slave-kznw"
server_key: "053658eb96f4939a2d9ccb1c4f1954a8"
server_key_iv: "6d58b042de161a0de5b789cae50f81cf"
server_key_version: "kuduclusterkey@0"
I20260925 02:45:28.230523 26345 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260925 02:45:28.245638 26360 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:45:28.247306 26345 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 02:45:28.247465 26345 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/master-0/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/master-0/wal
uuid: "28497343617d4cf396f60d556f7be937"
format_stamp: "Formatted at 2026-09-25 02:45:28 on dist-test-slave-kznw"
server_key: "053658eb96f4939a2d9ccb1c4f1954a8"
server_key_iv: "6d58b042de161a0de5b789cae50f81cf"
server_key_version: "kuduclusterkey@0"
I20260925 02:45:28.247617 26345 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:45:28.260946 26345 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:45:28.264142 26345 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:45:28.264341 26345 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:45:28.273020 26413 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.190:40793 every 8 connection(s)
I20260925 02:45:28.273025 26345 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.190:40793
I20260925 02:45:28.274358 26345 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
I20260925 02:45:28.277196 26414 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:45:28.279748 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 26345
I20260925 02:45:28.279879 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/master-0/wal/instance
I20260925 02:45:28.280218 13942 external_mini_cluster.cc:1428] Setting key 2f1c72c1bcdeb9b007b6e13665337e82
I20260925 02:45:28.283327 26414 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28497343617d4cf396f60d556f7be937: Bootstrap starting.
I20260925 02:45:28.285778 26414 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 28497343617d4cf396f60d556f7be937: Neither blocks nor log segments found. Creating new log.
I20260925 02:45:28.286496 26414 log.cc:824] T 00000000000000000000000000000000 P 28497343617d4cf396f60d556f7be937: Log is configured to *not* fsync() on all Append() calls
I20260925 02:45:28.288606 26414 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28497343617d4cf396f60d556f7be937: No bootstrap required, opened a new log
Sep 25 02:45:28 dist-test-slave-kznw krb5kdc[25896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304327, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.157.190@KRBTEST.COM
I20260925 02:45:28.292655 26414 raft_consensus.cc:348] T 00000000000000000000000000000000 P 28497343617d4cf396f60d556f7be937 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28497343617d4cf396f60d556f7be937" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 40793 } }
I20260925 02:45:28.292915 26414 raft_consensus.cc:374] T 00000000000000000000000000000000 P 28497343617d4cf396f60d556f7be937 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:45:28.292970 26414 raft_consensus.cc:729] T 00000000000000000000000000000000 P 28497343617d4cf396f60d556f7be937 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28497343617d4cf396f60d556f7be937, State: Initialized, Role: FOLLOWER
I20260925 02:45:28.293408 26414 consensus_queue.cc:260] T 00000000000000000000000000000000 P 28497343617d4cf396f60d556f7be937 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28497343617d4cf396f60d556f7be937" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 40793 } }
I20260925 02:45:28.293538 26414 raft_consensus.cc:388] T 00000000000000000000000000000000 P 28497343617d4cf396f60d556f7be937 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 02:45:28.293602 26414 raft_consensus.cc:482] T 00000000000000000000000000000000 P 28497343617d4cf396f60d556f7be937 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 02:45:28.293701 26414 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 28497343617d4cf396f60d556f7be937 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:45:28.294729 26414 raft_consensus.cc:504] T 00000000000000000000000000000000 P 28497343617d4cf396f60d556f7be937 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28497343617d4cf396f60d556f7be937" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 40793 } }
I20260925 02:45:28.295094 26414 leader_election.cc:302] T 00000000000000000000000000000000 P 28497343617d4cf396f60d556f7be937 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 28497343617d4cf396f60d556f7be937; no voters:
I20260925 02:45:28.295461 26414 leader_election.cc:288] T 00000000000000000000000000000000 P 28497343617d4cf396f60d556f7be937 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 02:45:28.295554 26419 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 28497343617d4cf396f60d556f7be937 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:45:28.295797 26419 raft_consensus.cc:686] T 00000000000000000000000000000000 P 28497343617d4cf396f60d556f7be937 [term 1 LEADER]: Becoming Leader. State: Replica: 28497343617d4cf396f60d556f7be937, State: Running, Role: LEADER
I20260925 02:45:28.296113 26419 consensus_queue.cc:237] T 00000000000000000000000000000000 P 28497343617d4cf396f60d556f7be937 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28497343617d4cf396f60d556f7be937" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 40793 } }
I20260925 02:45:28.296837 26414 sys_catalog.cc:564] T 00000000000000000000000000000000 P 28497343617d4cf396f60d556f7be937 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:45:28.297555 26417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:28.281707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40820 (local address 127.13.157.190:40793)
0925 02:45:28.282194 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0925 02:45:28.282213 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0925 02:45:28.282259 (+ 46us) server_negotiation.cc:407] Connection header received
0925 02:45:28.283120 (+ 861us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:45:28.283154 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:45:28.283582 (+ 428us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:45:28.283942 (+ 360us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:45:28.284818 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:28.286374 (+ 1556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:28.287434 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:28.288138 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:28.288306 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:45:28.291358 (+ 3052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:45:28.291365 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:45:28.291400 (+ 35us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:45:28.291439 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:45:28.293430 (+ 1991us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:45:28.294019 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:45:28.294025 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:45:28.294032 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:45:28.294183 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:45:28.294479 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:45:28.294482 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:45:28.294484 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:45:28.295045 (+ 561us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:45:28.295244 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0925 02:45:28.295791 (+ 547us) server_negotiation.cc:299] Negotiation successful
0925 02:45:28.296136 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":304,"thread_start_us":127,"threads_started":1}
I20260925 02:45:28.297556 26419 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28497343617d4cf396f60d556f7be937 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "28497343617d4cf396f60d556f7be937" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28497343617d4cf396f60d556f7be937" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 40793 } } }
I20260925 02:45:28.298164 26419 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28497343617d4cf396f60d556f7be937 [sys.catalog]: This master's current role is: LEADER
I20260925 02:45:28.298363 26419 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28497343617d4cf396f60d556f7be937 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 28497343617d4cf396f60d556f7be937. Latest consensus state: current_term: 1 leader_uuid: "28497343617d4cf396f60d556f7be937" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28497343617d4cf396f60d556f7be937" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 40793 } } }
I20260925 02:45:28.298440 26419 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28497343617d4cf396f60d556f7be937 [sys.catalog]: This master's current role is: LEADER
I20260925 02:45:28.299070 26414 ranger_client.cc:318] Using new properties file: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260925 02:45:28.303608 26428 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:45:28.307318 26428 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:45:28.316241 26428 catalog_manager.cc:1347] Generated new cluster ID: 3e68a71bc0fd4bf59edca2169e5a3cb4
I20260925 02:45:28.316309 26428 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 02:45:28.328904 26428 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 02:45:28.330175 26428 catalog_manager.cc:1504] Loading token signing keys...
I20260925 02:45:28.338922 26428 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 28497343617d4cf396f60d556f7be937: Generated new TSK 0
I20260925 02:45:28.339826 26428 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.157.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.157.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.157.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.129.keytab.
Entry for principal kudu/127.13.157.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.129.keytab.
I20260925 02:45:29.038010 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.157.148:42843/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.157.129:0
--local_ip_for_outbound_sockets=127.13.157.129
--webserver_interface=127.13.157.129
--webserver_port=0
--tserver_master_addrs=127.13.157.190:40793
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.129.keytab
--principal=kudu/127.13.157.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:35221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:45:29.194602 26459 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:45:29.194862 26459 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:45:29.194929 26459 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:45:29.199380 26459 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:45:29.199460 26459 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:45:29.199541 26459 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.157.129
I20260925 02:45:29.205241 26459 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:35221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.13.157.148:42843/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.129.keytab
--principal=kudu/127.13.157.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.157.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.157.129
--webserver_port=0
--tserver_master_addrs=127.13.157.190:40793
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26459
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.157.129
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:45:29.206456 26459 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:45:29.207794 26459 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:45:29.219074 26466 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:45:29.219167 26467 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:45:29.219128 26469 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:45:29.219571 26459 server_base.cc:1034] running on GCE node
I20260925 02:45:29.220091 26459 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:45:29.220912 26459 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:45:29.222108 26459 hybrid_clock.cc:648] HybridClock initialized: now 1790304329222082 us; error 33 us; skew 500 ppm
Sep 25 02:45:29 dist-test-slave-kznw krb5kdc[25896](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304329, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:45:29.225672 26459 init.cc:377] Logged in from keytab as kudu/127.13.157.129@KRBTEST.COM (short username kudu)
I20260925 02:45:29.226850 26459 webserver.cc:460] Webserver started at http://127.13.157.129:40615/ using document root <none> and password file <none>
I20260925 02:45:29.227592 26459 fs_manager.cc:359] Metadata directory not provided
I20260925 02:45:29.227694 26459 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:45:29.227982 26459 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 25 02:45:29 dist-test-slave-kznw krb5kdc[25896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304329, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.129@KRBTEST.COM for HTTP/127.13.157.148@KRBTEST.COM
I20260925 02:45:29.245252 26459 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/ts-0/data/instance:
uuid: "1bab1df9399a4603be7fec4253ce9b58"
format_stamp: "Formatted at 2026-09-25 02:45:29 on dist-test-slave-kznw"
server_key: "621a89efa9c454bbe6a7b9435100e055"
server_key_iv: "33c99ffff888fb1a925a8d3ecff329ba"
server_key_version: "kuduclusterkey@0"
I20260925 02:45:29.246169 26459 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/ts-0/wal/instance:
uuid: "1bab1df9399a4603be7fec4253ce9b58"
format_stamp: "Formatted at 2026-09-25 02:45:29 on dist-test-slave-kznw"
server_key: "621a89efa9c454bbe6a7b9435100e055"
server_key_iv: "33c99ffff888fb1a925a8d3ecff329ba"
server_key_version: "kuduclusterkey@0"
I20260925 02:45:29.251789 26459 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260925 02:45:29.270520 26476 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:45:29.272260 26459 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.002s
I20260925 02:45:29.272385 26459 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/ts-0/wal
uuid: "1bab1df9399a4603be7fec4253ce9b58"
format_stamp: "Formatted at 2026-09-25 02:45:29 on dist-test-slave-kznw"
server_key: "621a89efa9c454bbe6a7b9435100e055"
server_key_iv: "33c99ffff888fb1a925a8d3ecff329ba"
server_key_version: "kuduclusterkey@0"
I20260925 02:45:29.272514 26459 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:45:29.332612 26459 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:45:29.336366 26459 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:45:29.336621 26459 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:45:29.337355 26459 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:45:29.338450 26459 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:45:29.338555 26459 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:45:29.338671 26459 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:45:29.338752 26459 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:45:29.352298 26459 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.129:34073
I20260925 02:45:29.352603 26592 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.129:34073 every 8 connection(s)
I20260925 02:45:29.353699 26459 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/ts-0/data/info.pb
I20260925 02:45:29.357523 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 26459
I20260925 02:45:29.357667 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/ts-0/wal/instance
I20260925 02:45:29.358012 13942 external_mini_cluster.cc:1428] Setting key 4830a3c583ee7e91cc8d93697b2aca7f
WARNING: no policy specified for kudu/127.13.157.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.157.130@KRBTEST.COM" created.
Sep 25 02:45:29 dist-test-slave-kznw krb5kdc[25896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304329, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.129@KRBTEST.COM for kudu/127.13.157.190@KRBTEST.COM
I20260925 02:45:29.380553 26595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:29.356011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.129:36799 (local address 127.13.157.190:40793)
0925 02:45:29.356265 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0925 02:45:29.356270 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:45:29.357215 (+ 945us) server_negotiation.cc:407] Connection header received
0925 02:45:29.359682 (+ 2467us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:45:29.359685 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:45:29.359761 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:45:29.359825 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:45:29.363559 (+ 3734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:29.364426 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:29.366165 (+ 1739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:29.366516 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:29.366594 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:45:29.375646 (+ 9052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:45:29.375651 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:45:29.375653 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:45:29.375686 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:45:29.377634 (+ 1948us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:45:29.378437 (+ 803us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:45:29.378441 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:45:29.378444 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:45:29.378535 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:45:29.379108 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:45:29.379112 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:45:29.379114 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:45:29.379396 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:45:29.379487 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0925 02:45:29.380231 (+ 744us) server_negotiation.cc:299] Negotiation successful
0925 02:45:29.380366 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":85,"threads_started":1}
I20260925 02:45:29.383121 26596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:29.356499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:40793 (local address 127.13.157.129:36799)
0925 02:45:29.357053 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:29.357110 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0925 02:45:29.359428 (+ 2318us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:45:29.362250 (+ 2822us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:45:29.362260 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:45:29.362633 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:45:29.363380 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:45:29.363401 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:45:29.364590 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:45:29.364594 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:45:29.366021 (+ 1427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:45:29.366033 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:45:29.366830 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:45:29.366834 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:45:29.367056 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:45:29.371108 (+ 4052us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:45:29.371131 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:45:29.375409 (+ 4278us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:45:29.377950 (+ 2541us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:45:29.377956 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:45:29.377973 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:45:29.378317 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:45:29.378645 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:45:29.378648 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:45:29.378650 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:45:29.378987 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:45:29.379571 (+ 584us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:45:29.379578 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:45:29.379935 (+ 357us) client_negotiation.cc:769] Sending connection context
0925 02:45:29.380974 (+ 1039us) client_negotiation.cc:240] Negotiation successful
0925 02:45:29.381410 (+ 436us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":313,"thread_start_us":148,"threads_started":1}
I20260925 02:45:29.384907 26593 heartbeater.cc:347] Connected to a master server at 127.13.157.190:40793
I20260925 02:45:29.385200 26593 heartbeater.cc:464] Registering TS with master...
I20260925 02:45:29.385881 26593 heartbeater.cc:511] Master 127.13.157.190:40793 requested a full tablet report, sending...
I20260925 02:45:29.387770 26377 ts_manager.cc:194] Registered new tserver with Master: 1bab1df9399a4603be7fec4253ce9b58 (127.13.157.129:34073)
I20260925 02:45:29.389333 26377 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.157.129@KRBTEST.COM'} at 127.13.157.129:36799
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.157.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.130.keytab.
Entry for principal kudu/127.13.157.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.130.keytab.
I20260925 02:45:29.393853 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.157.148:42843/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.157.130:0
--local_ip_for_outbound_sockets=127.13.157.130
--webserver_interface=127.13.157.130
--webserver_port=0
--tserver_master_addrs=127.13.157.190:40793
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.130.keytab
--principal=kudu/127.13.157.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:35221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 25 02:45:29 dist-test-slave-kznw krb5kdc[25896](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790304329, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260925 02:45:29.550047 26599 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:45:29.550522 26599 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:45:29.550607 26599 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:45:29.556824 26599 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:45:29.556907 26599 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:45:29.556996 26599 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.157.130
I20260925 02:45:29.564972 26599 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:35221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.13.157.148:42843/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.130.keytab
--principal=kudu/127.13.157.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.157.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.157.130
--webserver_port=0
--tserver_master_addrs=127.13.157.190:40793
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26599
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.157.130
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:45:29.566322 26599 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:45:29.567695 26599 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:45:29.580943 26606 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:45:29.581394 26599 server_base.cc:1034] running on GCE node
W20260925 02:45:29.581527 26605 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:45:29.580865 26608 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:45:29.582177 26599 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:45:29.583061 26599 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:45:29.584265 26599 hybrid_clock.cc:648] HybridClock initialized: now 1790304329584235 us; error 46 us; skew 500 ppm
Sep 25 02:45:29 dist-test-slave-kznw krb5kdc[25896](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304329, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:45:29.588064 26599 init.cc:377] Logged in from keytab as kudu/127.13.157.130@KRBTEST.COM (short username kudu)
I20260925 02:45:29.589206 26599 webserver.cc:460] Webserver started at http://127.13.157.130:34211/ using document root <none> and password file <none>
I20260925 02:45:29.589920 26599 fs_manager.cc:359] Metadata directory not provided
I20260925 02:45:29.589991 26599 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:45:29.590234 26599 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 25 02:45:29 dist-test-slave-kznw krb5kdc[25896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304329, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.130@KRBTEST.COM for HTTP/127.13.157.148@KRBTEST.COM
I20260925 02:45:29.614612 26599 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/ts-1/data/instance:
uuid: "22006d4c201a4713bce30c888fe40b5b"
format_stamp: "Formatted at 2026-09-25 02:45:29 on dist-test-slave-kznw"
server_key: "219eb208fd416b10e12744ea4c1ee229"
server_key_iv: "1887caf9555c65b92bb0f735c30de2c0"
server_key_version: "kuduclusterkey@0"
I20260925 02:45:29.615448 26599 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/ts-1/wal/instance:
uuid: "22006d4c201a4713bce30c888fe40b5b"
format_stamp: "Formatted at 2026-09-25 02:45:29 on dist-test-slave-kznw"
server_key: "219eb208fd416b10e12744ea4c1ee229"
server_key_iv: "1887caf9555c65b92bb0f735c30de2c0"
server_key_version: "kuduclusterkey@0"
I20260925 02:45:29.620661 26599 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260925 02:45:29.651003 26615 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:45:29.653075 26599 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.001s
I20260925 02:45:29.653235 26599 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/ts-1/wal
uuid: "22006d4c201a4713bce30c888fe40b5b"
format_stamp: "Formatted at 2026-09-25 02:45:29 on dist-test-slave-kznw"
server_key: "219eb208fd416b10e12744ea4c1ee229"
server_key_iv: "1887caf9555c65b92bb0f735c30de2c0"
server_key_version: "kuduclusterkey@0"
I20260925 02:45:29.653401 26599 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:45:29.670109 26599 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:45:29.673769 26599 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:45:29.673974 26599 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:45:29.674705 26599 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:45:29.675840 26599 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:45:29.675904 26599 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:45:29.675967 26599 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:45:29.675997 26599 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:45:29.686926 26599 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.130:37541
I20260925 02:45:29.686995 26728 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.130:37541 every 8 connection(s)
I20260925 02:45:29.688099 26599 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/ts-1/data/info.pb
I20260925 02:45:29.697351 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 26599
I20260925 02:45:29.697506 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/ts-1/wal/instance
I20260925 02:45:29.697865 13942 external_mini_cluster.cc:1428] Setting key 0bb49822d76b413acb0d6ec06634c803
Sep 25 02:45:29 dist-test-slave-kznw krb5kdc[25896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304329, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.130@KRBTEST.COM for kudu/127.13.157.190@KRBTEST.COM
I20260925 02:45:29.707480 26595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:29.692538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.130:56031 (local address 127.13.157.190:40793)
0925 02:45:29.692675 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0925 02:45:29.692680 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:45:29.692953 (+ 273us) server_negotiation.cc:407] Connection header received
0925 02:45:29.694298 (+ 1345us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:45:29.694303 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:45:29.694365 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:45:29.694421 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:45:29.696562 (+ 2141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:29.697422 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:29.699582 (+ 2160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:29.699897 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:29.699984 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:45:29.703103 (+ 3119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:45:29.703107 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:45:29.703110 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:45:29.703141 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:45:29.704867 (+ 1726us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:45:29.705558 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:45:29.705562 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:45:29.705564 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:45:29.705656 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:45:29.706068 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:45:29.706071 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:45:29.706073 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:45:29.706315 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:45:29.706395 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0925 02:45:29.707142 (+ 747us) server_negotiation.cc:299] Negotiation successful
0925 02:45:29.707301 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260925 02:45:29.709229 26731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:29.691393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:40793 (local address 127.13.157.130:56031)
0925 02:45:29.692770 (+ 1377us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:29.692832 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0925 02:45:29.694004 (+ 1172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:45:29.694873 (+ 869us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:45:29.694883 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:45:29.695609 (+ 726us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:45:29.696355 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:45:29.696378 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:45:29.697681 (+ 1303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:45:29.697684 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:45:29.699438 (+ 1754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:45:29.699451 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:45:29.700232 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:45:29.700235 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:45:29.700447 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:45:29.701131 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:45:29.701154 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:45:29.702860 (+ 1706us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:45:29.705044 (+ 2184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:45:29.705050 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:45:29.705070 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:45:29.705436 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:45:29.705765 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:45:29.705769 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:45:29.705771 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:45:29.705957 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:45:29.706464 (+ 507us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:45:29.706471 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:45:29.706822 (+ 351us) client_negotiation.cc:769] Sending connection context
0925 02:45:29.707678 (+ 856us) client_negotiation.cc:240] Negotiation successful
0925 02:45:29.708051 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1154,"thread_start_us":141,"threads_started":1}
I20260925 02:45:29.710853 26729 heartbeater.cc:347] Connected to a master server at 127.13.157.190:40793
I20260925 02:45:29.711171 26729 heartbeater.cc:464] Registering TS with master...
I20260925 02:45:29.711855 26729 heartbeater.cc:511] Master 127.13.157.190:40793 requested a full tablet report, sending...
I20260925 02:45:29.713151 26377 ts_manager.cc:194] Registered new tserver with Master: 22006d4c201a4713bce30c888fe40b5b (127.13.157.130:37541)
I20260925 02:45:29.713963 26377 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.157.130@KRBTEST.COM'} at 127.13.157.130:56031
WARNING: no policy specified for kudu/127.13.157.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.157.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.157.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.131.keytab.
Entry for principal kudu/127.13.157.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.131.keytab.
I20260925 02:45:29.742095 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.157.148:42843/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.157.131:0
--local_ip_for_outbound_sockets=127.13.157.131
--webserver_interface=127.13.157.131
--webserver_port=0
--tserver_master_addrs=127.13.157.190:40793
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.131.keytab
--principal=kudu/127.13.157.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:35221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 25 02:45:29 dist-test-slave-kznw krb5kdc[25896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304329, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.190@KRBTEST.COM for HTTP/127.13.157.148@KRBTEST.COM
W20260925 02:45:29.901877 26734 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:45:29.902386 26734 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:45:29.902500 26734 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:45:29.909323 26734 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:45:29.909415 26734 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:45:29.909502 26734 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.157.131
I20260925 02:45:29.917418 26734 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:35221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.13.157.148:42843/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.131.keytab
--principal=kudu/127.13.157.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.157.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.157.131
--webserver_port=0
--tserver_master_addrs=127.13.157.190:40793
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26734
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.157.131
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:45:29.918787 26734 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:45:29.920187 26734 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:45:29.932928 26740 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:45:29.932943 26739 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:45:29.934175 26734 server_base.cc:1034] running on GCE node
W20260925 02:45:29.935390 26742 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:45:29.935953 26734 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:45:29.936810 26734 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:45:29.944223 26734 hybrid_clock.cc:648] HybridClock initialized: now 1790304329944128 us; error 108 us; skew 500 ppm
Sep 25 02:45:29 dist-test-slave-kznw krb5kdc[25896](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304329, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:45:29.948238 26734 init.cc:377] Logged in from keytab as kudu/127.13.157.131@KRBTEST.COM (short username kudu)
I20260925 02:45:29.949491 26734 webserver.cc:460] Webserver started at http://127.13.157.131:42357/ using document root <none> and password file <none>
I20260925 02:45:29.950314 26734 fs_manager.cc:359] Metadata directory not provided
I20260925 02:45:29.950466 26734 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:45:29.950757 26734 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 25 02:45:29 dist-test-slave-kznw krb5kdc[25896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304329, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.131@KRBTEST.COM for HTTP/127.13.157.148@KRBTEST.COM
I20260925 02:45:29.978376 26734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/ts-2/data/instance:
uuid: "8a0747e35bdc45f5972ca95a228aad98"
format_stamp: "Formatted at 2026-09-25 02:45:29 on dist-test-slave-kznw"
server_key: "46c18b0d6afcaad3cf7f99739d4db277"
server_key_iv: "e1d89e4719e8b1146cef1e2941fc32cb"
server_key_version: "kuduclusterkey@0"
I20260925 02:45:29.979193 26734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/ts-2/wal/instance:
uuid: "8a0747e35bdc45f5972ca95a228aad98"
format_stamp: "Formatted at 2026-09-25 02:45:29 on dist-test-slave-kznw"
server_key: "46c18b0d6afcaad3cf7f99739d4db277"
server_key_iv: "e1d89e4719e8b1146cef1e2941fc32cb"
server_key_version: "kuduclusterkey@0"
I20260925 02:45:29.998319 26734 fs_manager.cc:691] Time spent creating directory manager: real 0.019s user 0.003s sys 0.001s
I20260925 02:45:30.037612 26749 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:45:30.040599 26734 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260925 02:45:30.041338 26734 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/ts-2/wal
uuid: "8a0747e35bdc45f5972ca95a228aad98"
format_stamp: "Formatted at 2026-09-25 02:45:29 on dist-test-slave-kznw"
server_key: "46c18b0d6afcaad3cf7f99739d4db277"
server_key_iv: "e1d89e4719e8b1146cef1e2941fc32cb"
server_key_version: "kuduclusterkey@0"
I20260925 02:45:30.041662 26734 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:45:30.081012 26734 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:45:30.084151 26734 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:45:30.084345 26734 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:45:30.085007 26734 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:45:30.086156 26734 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:45:30.086212 26734 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:45:30.086270 26734 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:45:30.086297 26734 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:45:30.096660 26734 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.131:40941
I20260925 02:45:30.096660 26863 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.131:40941 every 8 connection(s)
I20260925 02:45:30.097802 26734 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/ts-2/data/info.pb
I20260925 02:45:30.098721 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 26734
I20260925 02:45:30.098845 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/ts-2/wal/instance
I20260925 02:45:30.099179 13942 external_mini_cluster.cc:1428] Setting key 6ceba12740d680f9e555b359b767985d
Sep 25 02:45:30 dist-test-slave-kznw krb5kdc[25896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304329, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.131@KRBTEST.COM for kudu/127.13.157.190@KRBTEST.COM
I20260925 02:45:30.115332 26595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:30.102854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.131:60753 (local address 127.13.157.190:40793)
0925 02:45:30.102987 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0925 02:45:30.102991 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:45:30.103005 (+ 14us) server_negotiation.cc:407] Connection header received
0925 02:45:30.103801 (+ 796us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:45:30.103804 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:45:30.103860 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:45:30.103930 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:45:30.105045 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:30.105580 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:30.106788 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:30.107092 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:30.107205 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:45:30.110197 (+ 2992us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:45:30.110201 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:45:30.110204 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:45:30.110235 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:45:30.112039 (+ 1804us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:45:30.112703 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:45:30.112707 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:45:30.112709 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:45:30.112805 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:45:30.113551 (+ 746us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:45:30.113554 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:45:30.113556 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:45:30.113805 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:45:30.113884 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0925 02:45:30.114618 (+ 734us) server_negotiation.cc:299] Negotiation successful
0925 02:45:30.114762 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260925 02:45:30.117586 26866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:30.101744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:40793 (local address 127.13.157.131:60753)
0925 02:45:30.102614 (+ 870us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:30.102684 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0925 02:45:30.103583 (+ 899us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:45:30.104107 (+ 524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:45:30.104115 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:45:30.104397 (+ 282us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:45:30.104903 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:45:30.104917 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:45:30.105705 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:45:30.105708 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:45:30.106674 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:45:30.106683 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:45:30.107384 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:45:30.107388 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:45:30.107586 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:45:30.108274 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:45:30.108296 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:45:30.109979 (+ 1683us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:45:30.112202 (+ 2223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:45:30.112209 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:45:30.112228 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:45:30.112583 (+ 355us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:45:30.112917 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:45:30.112920 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:45:30.112922 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:45:30.113118 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:45:30.113972 (+ 854us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:45:30.113979 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:45:30.114336 (+ 357us) client_negotiation.cc:769] Sending connection context
0925 02:45:30.115750 (+ 1414us) client_negotiation.cc:240] Negotiation successful
0925 02:45:30.116774 (+ 1024us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":647,"thread_start_us":158,"threads_started":1}
I20260925 02:45:30.118774 26864 heartbeater.cc:347] Connected to a master server at 127.13.157.190:40793
I20260925 02:45:30.119065 26864 heartbeater.cc:464] Registering TS with master...
I20260925 02:45:30.119769 26864 heartbeater.cc:511] Master 127.13.157.190:40793 requested a full tablet report, sending...
I20260925 02:45:30.120909 26377 ts_manager.cc:194] Registered new tserver with Master: 8a0747e35bdc45f5972ca95a228aad98 (127.13.157.131:40941)
I20260925 02:45:30.121522 26377 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.157.131@KRBTEST.COM'} at 127.13.157.131:60753
I20260925 02:45:30.128141 13942 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 02:45:30.129032 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 26459
I20260925 02:45:30.137002 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 26599
I20260925 02:45:30.143196 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 26734
I20260925 02:45:30.151681 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 26345
I20260925 02:45:30.160025 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.157.190:40793
--webserver_interface=127.13.157.190
--webserver_port=42935
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab
--principal=kudu/127.13.157.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:35221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.157.190:40793
--ranger_config_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.157.148:42843/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.13.157.148:42843/kms with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:45:30.270712 26868 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:45:30.271016 26868 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:45:30.271070 26868 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:45:30.275068 26868 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:45:30.275121 26868 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:45:30.275138 26868 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:45:30.275154 26868 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:45:30.275182 26868 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:45:30.280586 26868 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:35221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.13.157.148:42843/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.13.157.190:40793
--ranger_config_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab
--principal=kudu/127.13.157.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.157.190:40793
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.157.190
--webserver_port=42935
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26868
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:45:30.281610 26868 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:45:30.282581 26868 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:45:30.290800 26876 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:45:30.290824 26873 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:45:30.290875 26874 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:45:30.291013 26868 server_base.cc:1034] running on GCE node
I20260925 02:45:30.291723 26868 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:45:30.292858 26868 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:45:30.294030 26868 hybrid_clock.cc:648] HybridClock initialized: now 1790304330293994 us; error 49 us; skew 500 ppm
Sep 25 02:45:30 dist-test-slave-kznw krb5kdc[25896](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304330, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:45:30.296816 26868 init.cc:377] Logged in from keytab as kudu/127.13.157.190@KRBTEST.COM (short username kudu)
I20260925 02:45:30.297732 26868 webserver.cc:460] Webserver started at http://127.13.157.190:42935/ using document root <none> and password file <none>
I20260925 02:45:30.298243 26868 fs_manager.cc:359] Metadata directory not provided
I20260925 02:45:30.298307 26868 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 25 02:45:30 dist-test-slave-kznw krb5kdc[25896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304330, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.190@KRBTEST.COM for HTTP/127.13.157.148@KRBTEST.COM
I20260925 02:45:30.358835 26868 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260925 02:45:30.361210 26884 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:45:30.362666 26868 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260925 02:45:30.362821 26868 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/master-0/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/master-0/wal
uuid: "28497343617d4cf396f60d556f7be937"
format_stamp: "Formatted at 2026-09-25 02:45:28 on dist-test-slave-kznw"
server_key: "053658eb96f4939a2d9ccb1c4f1954a8"
server_key_iv: "6d58b042de161a0de5b789cae50f81cf"
server_key_version: "kuduclusterkey@0"
I20260925 02:45:30.363413 26868 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:45:30.386823 26868 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:45:30.389683 26868 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:45:30.389868 26868 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:45:30.397049 26868 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.190:40793
I20260925 02:45:30.397089 26936 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.190:40793 every 8 connection(s)
I20260925 02:45:30.398123 26868 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
I20260925 02:45:30.402441 26937 sys_catalog.cc:263] Verifying existing consensus state
I20260925 02:45:30.404107 26937 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28497343617d4cf396f60d556f7be937: Bootstrap starting.
I20260925 02:45:30.407147 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 26868
I20260925 02:45:30.407555 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.157.148:42843/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.157.129:34073
--local_ip_for_outbound_sockets=127.13.157.129
--tserver_master_addrs=127.13.157.190:40793
--webserver_port=40615
--webserver_interface=127.13.157.129
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.129.keytab
--principal=kudu/127.13.157.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:35221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.13.157.148:42843/kms with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 02:45:30.421180 26937 log.cc:824] T 00000000000000000000000000000000 P 28497343617d4cf396f60d556f7be937: Log is configured to *not* fsync() on all Append() calls
I20260925 02:45:30.428185 26937 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28497343617d4cf396f60d556f7be937: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 02:45:30.428560 26937 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28497343617d4cf396f60d556f7be937: Bootstrap complete.
I20260925 02:45:30.432612 26937 raft_consensus.cc:348] T 00000000000000000000000000000000 P 28497343617d4cf396f60d556f7be937 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28497343617d4cf396f60d556f7be937" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 40793 } }
I20260925 02:45:30.433025 26937 raft_consensus.cc:729] T 00000000000000000000000000000000 P 28497343617d4cf396f60d556f7be937 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28497343617d4cf396f60d556f7be937, State: Initialized, Role: FOLLOWER
I20260925 02:45:30.433504 26937 consensus_queue.cc:260] T 00000000000000000000000000000000 P 28497343617d4cf396f60d556f7be937 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28497343617d4cf396f60d556f7be937" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 40793 } }
I20260925 02:45:30.433615 26937 raft_consensus.cc:388] T 00000000000000000000000000000000 P 28497343617d4cf396f60d556f7be937 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 02:45:30.433677 26937 raft_consensus.cc:482] T 00000000000000000000000000000000 P 28497343617d4cf396f60d556f7be937 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 02:45:30.433777 26937 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 28497343617d4cf396f60d556f7be937 [term 1 FOLLOWER]: Advancing to term 2
I20260925 02:45:30.435719 26937 raft_consensus.cc:504] T 00000000000000000000000000000000 P 28497343617d4cf396f60d556f7be937 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28497343617d4cf396f60d556f7be937" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 40793 } }
I20260925 02:45:30.436028 26937 leader_election.cc:302] T 00000000000000000000000000000000 P 28497343617d4cf396f60d556f7be937 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 28497343617d4cf396f60d556f7be937; no voters:
I20260925 02:45:30.436313 26937 leader_election.cc:288] T 00000000000000000000000000000000 P 28497343617d4cf396f60d556f7be937 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260925 02:45:30.436420 26941 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 28497343617d4cf396f60d556f7be937 [term 2 FOLLOWER]: Leader election won for term 2
I20260925 02:45:30.436625 26941 raft_consensus.cc:686] T 00000000000000000000000000000000 P 28497343617d4cf396f60d556f7be937 [term 2 LEADER]: Becoming Leader. State: Replica: 28497343617d4cf396f60d556f7be937, State: Running, Role: LEADER
I20260925 02:45:30.436919 26941 consensus_queue.cc:237] T 00000000000000000000000000000000 P 28497343617d4cf396f60d556f7be937 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28497343617d4cf396f60d556f7be937" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 40793 } }
I20260925 02:45:30.437582 26937 sys_catalog.cc:564] T 00000000000000000000000000000000 P 28497343617d4cf396f60d556f7be937 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:45:30.437878 26941 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28497343617d4cf396f60d556f7be937 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 28497343617d4cf396f60d556f7be937. Latest consensus state: current_term: 2 leader_uuid: "28497343617d4cf396f60d556f7be937" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28497343617d4cf396f60d556f7be937" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 40793 } } }
I20260925 02:45:30.438067 26941 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28497343617d4cf396f60d556f7be937 [sys.catalog]: This master's current role is: LEADER
I20260925 02:45:30.438687 26942 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28497343617d4cf396f60d556f7be937 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "28497343617d4cf396f60d556f7be937" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28497343617d4cf396f60d556f7be937" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 40793 } } }
I20260925 02:45:30.438845 26942 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28497343617d4cf396f60d556f7be937 [sys.catalog]: This master's current role is: LEADER
I20260925 02:45:30.439517 26945 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:45:30.439895 26937 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260925 02:45:30.445827 26945 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:45:30.446645 26945 catalog_manager.cc:1259] Loaded cluster ID: 3e68a71bc0fd4bf59edca2169e5a3cb4
I20260925 02:45:30.446702 26945 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 02:45:30.449355 26945 catalog_manager.cc:1504] Loading token signing keys...
I20260925 02:45:30.450389 26945 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 28497343617d4cf396f60d556f7be937: Loaded TSK: 0
I20260925 02:45:30.451674 26945 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260925 02:45:30.547885 26939 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:45:30.548243 26939 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:45:30.548306 26939 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:45:30.554335 26939 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:45:30.554421 26939 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:45:30.554509 26939 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.157.129
I20260925 02:45:30.562363 26939 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:35221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.13.157.148:42843/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.129.keytab
--principal=kudu/127.13.157.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.157.129:34073
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.157.129
--webserver_port=40615
--tserver_master_addrs=127.13.157.190:40793
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26939
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.157.129
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:45:30.563714 26939 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:45:30.565037 26939 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:45:30.577854 26977 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:45:30.578025 26975 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:45:30.578271 26974 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:45:30.578577 26939 server_base.cc:1034] running on GCE node
I20260925 02:45:30.579124 26939 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:45:30.579984 26939 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:45:30.581172 26939 hybrid_clock.cc:648] HybridClock initialized: now 1790304330581151 us; error 54 us; skew 500 ppm
Sep 25 02:45:30 dist-test-slave-kznw krb5kdc[25896](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304330, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:45:30.584862 26939 init.cc:377] Logged in from keytab as kudu/127.13.157.129@KRBTEST.COM (short username kudu)
I20260925 02:45:30.586038 26939 webserver.cc:460] Webserver started at http://127.13.157.129:40615/ using document root <none> and password file <none>
I20260925 02:45:30.586725 26939 fs_manager.cc:359] Metadata directory not provided
I20260925 02:45:30.586834 26939 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 25 02:45:30 dist-test-slave-kznw krb5kdc[25896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304330, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.129@KRBTEST.COM for HTTP/127.13.157.148@KRBTEST.COM
I20260925 02:45:30.616533 26939 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.003s sys 0.000s
I20260925 02:45:30.625247 26985 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:45:30.627315 26939 fs_manager.cc:725] Time spent opening block manager: real 0.009s user 0.003s sys 0.000s
I20260925 02:45:30.627552 26939 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/ts-0/wal
uuid: "1bab1df9399a4603be7fec4253ce9b58"
format_stamp: "Formatted at 2026-09-25 02:45:29 on dist-test-slave-kznw"
server_key: "621a89efa9c454bbe6a7b9435100e055"
server_key_iv: "33c99ffff888fb1a925a8d3ecff329ba"
server_key_version: "kuduclusterkey@0"
I20260925 02:45:30.628463 26939 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:45:30.662997 26939 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:45:30.668363 26939 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:45:30.668568 26939 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:45:30.669204 26939 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:45:30.670183 26939 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:45:30.670233 26939 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:45:30.670287 26939 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:45:30.670312 26939 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:45:30.680616 26939 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.129:34073
I20260925 02:45:30.680922 27098 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.129:34073 every 8 connection(s)
I20260925 02:45:30.682312 26939 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/ts-0/data/info.pb
I20260925 02:45:30.685323 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 26939
I20260925 02:45:30.685838 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.157.148:42843/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.157.130:37541
--local_ip_for_outbound_sockets=127.13.157.130
--tserver_master_addrs=127.13.157.190:40793
--webserver_port=34211
--webserver_interface=127.13.157.130
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.130.keytab
--principal=kudu/127.13.157.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:35221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.13.157.148:42843/kms with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 25 02:45:30 dist-test-slave-kznw krb5kdc[25896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304330, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.129@KRBTEST.COM for kudu/127.13.157.190@KRBTEST.COM
I20260925 02:45:30.705170 27101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:30.684785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:40793 (local address 127.13.157.129:55313)
0925 02:45:30.687156 (+ 2371us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:30.687223 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0925 02:45:30.688034 (+ 811us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:45:30.689417 (+ 1383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:45:30.689426 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:45:30.689758 (+ 332us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:45:30.690369 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:45:30.690385 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:45:30.692515 (+ 2130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:45:30.692518 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:45:30.695129 (+ 2611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:45:30.695143 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:45:30.696377 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:45:30.696382 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:45:30.696595 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:45:30.697290 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:45:30.697312 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:45:30.698944 (+ 1632us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:45:30.701519 (+ 2575us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:45:30.701526 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:45:30.701542 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:45:30.701922 (+ 380us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:45:30.702332 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:45:30.702336 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:45:30.702338 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:45:30.702532 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:45:30.703329 (+ 797us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:45:30.703336 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:45:30.703629 (+ 293us) client_negotiation.cc:769] Sending connection context
0925 02:45:30.703828 (+ 199us) client_negotiation.cc:240] Negotiation successful
0925 02:45:30.704151 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2160,"thread_start_us":124,"threads_started":1}
I20260925 02:45:30.706946 27102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:30.686331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.129:55313 (local address 127.13.157.190:40793)
0925 02:45:30.686839 (+ 508us) server_negotiation.cc:206] Beginning negotiation
0925 02:45:30.686862 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0925 02:45:30.687382 (+ 520us) server_negotiation.cc:407] Connection header received
0925 02:45:30.688494 (+ 1112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:45:30.688511 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:45:30.688920 (+ 409us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:45:30.691086 (+ 2166us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:45:30.691348 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:30.692381 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:30.695293 (+ 2912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:30.696179 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:30.696362 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:45:30.699201 (+ 2839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:45:30.699209 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:45:30.699229 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:45:30.699267 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:45:30.701362 (+ 2095us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:45:30.702067 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:45:30.702074 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:45:30.702080 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:45:30.702227 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:45:30.702630 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:45:30.702633 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:45:30.702635 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:45:30.703146 (+ 511us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:45:30.703272 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0925 02:45:30.705746 (+ 2474us) server_negotiation.cc:299] Negotiation successful
0925 02:45:30.706046 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":110,"threads_started":1}
I20260925 02:45:30.710317 27099 heartbeater.cc:347] Connected to a master server at 127.13.157.190:40793
I20260925 02:45:30.710537 27099 heartbeater.cc:464] Registering TS with master...
I20260925 02:45:30.711094 27099 heartbeater.cc:511] Master 127.13.157.190:40793 requested a full tablet report, sending...
W20260925 02:45:30.713265 27099 heartbeater.cc:644] Failed to heartbeat to 127.13.157.190:40793 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260925 02:45:30.713380 27099 heartbeater.cc:464] Registering TS with master...
I20260925 02:45:30.713538 27099 heartbeater.cc:511] Master 127.13.157.190:40793 requested a full tablet report, sending...
I20260925 02:45:30.714349 27099 heartbeater.cc:464] Registering TS with master...
I20260925 02:45:30.714509 27099 heartbeater.cc:511] Master 127.13.157.190:40793 requested a full tablet report, sending...
W20260925 02:45:30.715358 27099 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260925 02:45:30.819736 27103 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:45:30.820003 27103 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:45:30.820061 27103 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:45:30.825114 27103 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:45:30.825199 27103 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:45:30.825294 27103 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.157.130
I20260925 02:45:30.830865 27103 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:35221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.13.157.148:42843/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.130.keytab
--principal=kudu/127.13.157.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.157.130:37541
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.157.130
--webserver_port=34211
--tserver_master_addrs=127.13.157.190:40793
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27103
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.157.130
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:45:30.832011 27103 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:45:30.833377 27103 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:45:30.848942 27111 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:45:30.849300 27103 server_base.cc:1034] running on GCE node
W20260925 02:45:30.849490 27108 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:45:30.851366 27109 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:45:30.851889 27103 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:45:30.852731 27103 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:45:30.856040 27103 hybrid_clock.cc:648] HybridClock initialized: now 1790304330855997 us; error 59 us; skew 500 ppm
Sep 25 02:45:30 dist-test-slave-kznw krb5kdc[25896](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304330, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:45:30.860236 27103 init.cc:377] Logged in from keytab as kudu/127.13.157.130@KRBTEST.COM (short username kudu)
I20260925 02:45:30.861439 27103 webserver.cc:460] Webserver started at http://127.13.157.130:34211/ using document root <none> and password file <none>
I20260925 02:45:30.862169 27103 fs_manager.cc:359] Metadata directory not provided
I20260925 02:45:30.862282 27103 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 25 02:45:30 dist-test-slave-kznw krb5kdc[25896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304330, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.130@KRBTEST.COM for HTTP/127.13.157.148@KRBTEST.COM
I20260925 02:45:30.894002 27103 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260925 02:45:30.897527 27119 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:45:30.899535 27103 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260925 02:45:30.899695 27103 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/ts-1/wal
uuid: "22006d4c201a4713bce30c888fe40b5b"
format_stamp: "Formatted at 2026-09-25 02:45:29 on dist-test-slave-kznw"
server_key: "219eb208fd416b10e12744ea4c1ee229"
server_key_iv: "1887caf9555c65b92bb0f735c30de2c0"
server_key_version: "kuduclusterkey@0"
I20260925 02:45:30.900501 27103 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:45:30.929334 27103 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:45:30.933178 27103 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:45:30.933388 27103 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:45:30.934166 27103 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:45:30.935158 27103 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:45:30.935209 27103 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:45:30.935271 27103 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:45:30.935302 27103 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:45:30.946192 27103 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.130:37541
I20260925 02:45:30.947551 27103 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/ts-1/data/info.pb
I20260925 02:45:30.950107 27233 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.130:37541 every 8 connection(s)
I20260925 02:45:30.957491 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 27103
I20260925 02:45:30.957998 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.157.148:42843/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.157.131:40941
--local_ip_for_outbound_sockets=127.13.157.131
--tserver_master_addrs=127.13.157.190:40793
--webserver_port=42357
--webserver_interface=127.13.157.131
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.131.keytab
--principal=kudu/127.13.157.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:35221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.13.157.148:42843/kms with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 25 02:45:30 dist-test-slave-kznw krb5kdc[25896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304330, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.130@KRBTEST.COM for kudu/127.13.157.190@KRBTEST.COM
I20260925 02:45:30.963671 27102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:30.951756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.130:42217 (local address 127.13.157.190:40793)
0925 02:45:30.951919 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0925 02:45:30.951924 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:45:30.952824 (+ 900us) server_negotiation.cc:407] Connection header received
0925 02:45:30.953829 (+ 1005us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:45:30.953832 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:45:30.953894 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:45:30.953966 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:45:30.955060 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:30.955583 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:30.956917 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:30.957154 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:30.957213 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:45:30.959771 (+ 2558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:45:30.959776 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:45:30.959779 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:45:30.959809 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:45:30.961373 (+ 1564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:45:30.961956 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:45:30.961960 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:45:30.961961 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:45:30.962048 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:45:30.962414 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:45:30.962417 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:45:30.962419 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:45:30.962631 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:45:30.962714 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0925 02:45:30.963367 (+ 653us) server_negotiation.cc:299] Negotiation successful
0925 02:45:30.963501 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260925 02:45:30.964582 27236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:30.952179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:40793 (local address 127.13.157.130:42217)
0925 02:45:30.952670 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:30.952723 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0925 02:45:30.953623 (+ 900us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:45:30.954135 (+ 512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:45:30.954142 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:45:30.954416 (+ 274us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:45:30.954892 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:45:30.954938 (+ 46us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:45:30.955701 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:45:30.955704 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:45:30.956775 (+ 1071us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:45:30.956785 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:45:30.957392 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:45:30.957394 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:45:30.957541 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:45:30.958120 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:45:30.958142 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:45:30.959600 (+ 1458us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:45:30.961507 (+ 1907us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:45:30.961514 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:45:30.961529 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:45:30.961859 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:45:30.962136 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:45:30.962139 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:45:30.962141 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:45:30.962309 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:45:30.962775 (+ 466us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:45:30.962782 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:45:30.963095 (+ 313us) client_negotiation.cc:769] Sending connection context
0925 02:45:30.963289 (+ 194us) client_negotiation.cc:240] Negotiation successful
0925 02:45:30.963592 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":134,"threads_started":1}
I20260925 02:45:30.966046 27234 heartbeater.cc:347] Connected to a master server at 127.13.157.190:40793
I20260925 02:45:30.966291 27234 heartbeater.cc:464] Registering TS with master...
I20260925 02:45:30.966786 27234 heartbeater.cc:511] Master 127.13.157.190:40793 requested a full tablet report, sending...
W20260925 02:45:30.968534 27234 heartbeater.cc:644] Failed to heartbeat to 127.13.157.190:40793 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260925 02:45:30.968664 27234 heartbeater.cc:464] Registering TS with master...
I20260925 02:45:30.968858 27234 heartbeater.cc:511] Master 127.13.157.190:40793 requested a full tablet report, sending...
I20260925 02:45:30.969740 27234 heartbeater.cc:464] Registering TS with master...
I20260925 02:45:30.969915 27234 heartbeater.cc:511] Master 127.13.157.190:40793 requested a full tablet report, sending...
W20260925 02:45:30.970701 27234 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260925 02:45:31.093119 27237 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:45:31.093392 27237 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:45:31.093436 27237 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:45:31.098142 27237 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:45:31.098212 27237 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:45:31.098304 27237 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.157.131
I20260925 02:45:31.105031 27237 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:35221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.13.157.148:42843/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.131.keytab
--principal=kudu/127.13.157.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.157.131:40941
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.157.131
--webserver_port=42357
--tserver_master_addrs=127.13.157.190:40793
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27237
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.157.131
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:45:31.106441 27237 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:45:31.107800 27237 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:45:31.117961 27243 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:45:31.117975 27242 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:45:31.118245 27245 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:45:31.118458 27237 server_base.cc:1034] running on GCE node
I20260925 02:45:31.118829 27237 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:45:31.119504 27237 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:45:31.120677 27237 hybrid_clock.cc:648] HybridClock initialized: now 1790304331120643 us; error 39 us; skew 500 ppm
Sep 25 02:45:31 dist-test-slave-kznw krb5kdc[25896](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304331, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:45:31.124239 27237 init.cc:377] Logged in from keytab as kudu/127.13.157.131@KRBTEST.COM (short username kudu)
I20260925 02:45:31.125151 27237 webserver.cc:460] Webserver started at http://127.13.157.131:42357/ using document root <none> and password file <none>
I20260925 02:45:31.125686 27237 fs_manager.cc:359] Metadata directory not provided
I20260925 02:45:31.125769 27237 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 25 02:45:31 dist-test-slave-kznw krb5kdc[25896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304331, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.131@KRBTEST.COM for HTTP/127.13.157.148@KRBTEST.COM
I20260925 02:45:31.150596 27237 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260925 02:45:31.153164 27253 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:45:31.154485 27237 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260925 02:45:31.154599 27237 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/ts-2/wal
uuid: "8a0747e35bdc45f5972ca95a228aad98"
format_stamp: "Formatted at 2026-09-25 02:45:29 on dist-test-slave-kznw"
server_key: "46c18b0d6afcaad3cf7f99739d4db277"
server_key_iv: "e1d89e4719e8b1146cef1e2941fc32cb"
server_key_version: "kuduclusterkey@0"
I20260925 02:45:31.155192 27237 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:45:31.171911 27237 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:45:31.174791 27237 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:45:31.174999 27237 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:45:31.175633 27237 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:45:31.176534 27237 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:45:31.176584 27237 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:45:31.176640 27237 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:45:31.176668 27237 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:45:31.186630 27237 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.131:40941
I20260925 02:45:31.186751 27373 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.131:40941 every 8 connection(s)
I20260925 02:45:31.187799 27237 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/ts-2/data/info.pb
I20260925 02:45:31.196070 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 27237
Sep 25 02:45:31 dist-test-slave-kznw krb5kdc[25896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304331, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.131@KRBTEST.COM for kudu/127.13.157.190@KRBTEST.COM
I20260925 02:45:31.207512 27102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:31.189848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.131:45087 (local address 127.13.157.190:40793)
0925 02:45:31.190012 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0925 02:45:31.190017 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:45:31.190883 (+ 866us) server_negotiation.cc:407] Connection header received
0925 02:45:31.191874 (+ 991us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:45:31.191878 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:45:31.191939 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:45:31.192051 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:45:31.193461 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:31.194254 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:31.195459 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:31.195813 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:31.195939 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:45:31.198893 (+ 2954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:45:31.198896 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:45:31.198898 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:45:31.198953 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:45:31.204804 (+ 5851us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:45:31.205561 (+ 757us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:45:31.205565 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:45:31.205567 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:45:31.205687 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:45:31.206127 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:45:31.206130 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:45:31.206132 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:45:31.206401 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:45:31.206497 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0925 02:45:31.207201 (+ 704us) server_negotiation.cc:299] Negotiation successful
0925 02:45:31.207336 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260925 02:45:31.208774 27378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:31.196774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40830 (local address 127.13.157.190:40793)
0925 02:45:31.197475 (+ 701us) server_negotiation.cc:206] Beginning negotiation
0925 02:45:31.197483 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 02:45:31.197496 (+ 13us) server_negotiation.cc:407] Connection header received
0925 02:45:31.197528 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:45:31.197531 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:45:31.197573 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:45:31.197666 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:45:31.198526 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:31.199402 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:31.200589 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:31.200963 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:31.201066 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:45:31.202733 (+ 1667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:45:31.202736 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:45:31.202739 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:45:31.202768 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:45:31.204625 (+ 1857us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:45:31.205571 (+ 946us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:45:31.205575 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:45:31.205576 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:45:31.205716 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:45:31.206570 (+ 854us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:45:31.206573 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:45:31.206575 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:45:31.206767 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:45:31.206986 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0925 02:45:31.208482 (+ 1496us) server_negotiation.cc:299] Negotiation successful
0925 02:45:31.208608 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":580,"thread_start_us":100,"threads_started":1}
I20260925 02:45:31.209740 27376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:31.190225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:40793 (local address 127.13.157.131:45087)
0925 02:45:31.190707 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:31.190768 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0925 02:45:31.191649 (+ 881us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:45:31.192332 (+ 683us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:45:31.192342 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:45:31.192700 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:45:31.193313 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:45:31.193329 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:45:31.194394 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:45:31.194398 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:45:31.195344 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:45:31.195354 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:45:31.195994 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:45:31.195998 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:45:31.196192 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:45:31.197085 (+ 893us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:45:31.197101 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:45:31.198726 (+ 1625us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:45:31.204939 (+ 6213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:45:31.204946 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:45:31.204967 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:45:31.205449 (+ 482us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:45:31.205808 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:45:31.205811 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:45:31.205814 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:45:31.206027 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:45:31.206518 (+ 491us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:45:31.206526 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:45:31.206896 (+ 370us) client_negotiation.cc:769] Sending connection context
0925 02:45:31.207163 (+ 267us) client_negotiation.cc:240] Negotiation successful
0925 02:45:31.207536 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":126,"threads_started":1}
I20260925 02:45:31.211256 27374 heartbeater.cc:347] Connected to a master server at 127.13.157.190:40793
I20260925 02:45:31.211529 27374 heartbeater.cc:464] Registering TS with master...
I20260925 02:45:31.212131 27374 heartbeater.cc:511] Master 127.13.157.190:40793 requested a full tablet report, sending...
I20260925 02:45:31.213457 26901 ts_manager.cc:194] Registered new tserver with Master: 8a0747e35bdc45f5972ca95a228aad98 (127.13.157.131:40941)
I20260925 02:45:31.214644 26901 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.157.131@KRBTEST.COM'} at 127.13.157.131:45087
Sep 25 02:45:31 dist-test-slave-kznw krb5kdc[25896](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790304331, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:45:31.715587 27099 heartbeater.cc:464] Registering TS with master...
I20260925 02:45:31.715939 27099 heartbeater.cc:511] Master 127.13.157.190:40793 requested a full tablet report, sending...
I20260925 02:45:31.716828 26901 ts_manager.cc:194] Registered new tserver with Master: 1bab1df9399a4603be7fec4253ce9b58 (127.13.157.129:34073)
I20260925 02:45:31.717661 26901 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.157.129@KRBTEST.COM'} at 127.13.157.129:55313
Sep 25 02:45:31 dist-test-slave-kznw krb5kdc[25896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304331, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.190@KRBTEST.COM for HTTP/127.13.157.148@KRBTEST.COM
I20260925 02:45:31.970979 27234 heartbeater.cc:464] Registering TS with master...
I20260925 02:45:31.971307 27234 heartbeater.cc:511] Master 127.13.157.190:40793 requested a full tablet report, sending...
I20260925 02:45:31.972272 26901 ts_manager.cc:194] Registered new tserver with Master: 22006d4c201a4713bce30c888fe40b5b (127.13.157.130:37541)
I20260925 02:45:31.973099 26901 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.157.130@KRBTEST.COM'} at 127.13.157.130:42217
I20260925 02:45:31.974563 13942 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 02:45:31.983639 27388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:31.976203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39562 (local address 127.13.157.190:40793)
0925 02:45:31.976452 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0925 02:45:31.976457 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:45:31.976574 (+ 117us) server_negotiation.cc:407] Connection header received
0925 02:45:31.976691 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:45:31.976694 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:45:31.976738 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:45:31.976809 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:45:31.977453 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:31.977955 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:31.978744 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:31.979005 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:31.979092 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:45:31.980532 (+ 1440us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:45:31.980536 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:45:31.980539 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:45:31.980563 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:45:31.982083 (+ 1520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:45:31.982465 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:45:31.982469 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:45:31.982471 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:45:31.982547 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:45:31.982785 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:45:31.982789 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:45:31.982790 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:45:31.983001 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:45:31.983126 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0925 02:45:31.983339 (+ 213us) server_negotiation.cc:299] Negotiation successful
0925 02:45:31.983456 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":74,"threads_started":1}
I20260925 02:45:31.986845 26901 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39562:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 02:45:31.990191 26901 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 02:45:32.012415 27399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:32.002811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52072 (local address 127.13.157.129:34073)
0925 02:45:32.004802 (+ 1991us) server_negotiation.cc:206] Beginning negotiation
0925 02:45:32.004809 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 02:45:32.005337 (+ 528us) server_negotiation.cc:407] Connection header received
0925 02:45:32.005540 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:45:32.005545 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:45:32.005778 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:45:32.005930 (+ 152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:45:32.006424 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:32.007523 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:32.009923 (+ 2400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:32.010753 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:32.010849 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:45:32.010999 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0925 02:45:32.011399 (+ 400us) server_negotiation.cc:299] Negotiation successful
0925 02:45:32.011501 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1867,"thread_start_us":98,"threads_started":1}
I20260925 02:45:32.012908 27396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:32.002380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.129:34073 (local address 127.0.0.1:52072)
0925 02:45:32.005248 (+ 2868us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:32.005259 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 02:45:32.005400 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:45:32.006129 (+ 729us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:45:32.006135 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:45:32.006160 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:45:32.006288 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:45:32.006295 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:45:32.007662 (+ 1367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:45:32.007665 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:45:32.009775 (+ 2110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:45:32.009786 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:45:32.011087 (+ 1301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:45:32.011090 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:45:32.011189 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:45:32.011231 (+ 42us) client_negotiation.cc:769] Sending connection context
0925 02:45:32.012645 (+ 1414us) client_negotiation.cc:240] Negotiation successful
0925 02:45:32.012772 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2810,"thread_start_us":95,"threads_started":1}
I20260925 02:45:32.014808 27398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:32.000793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.130:37541 (local address 127.0.0.1:59250)
0925 02:45:32.002744 (+ 1951us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:32.002796 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0925 02:45:32.002950 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:45:32.007241 (+ 4291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:45:32.007244 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:45:32.007261 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:45:32.007346 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:45:32.007352 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:45:32.009952 (+ 2600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:45:32.009956 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:45:32.011041 (+ 1085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:45:32.011050 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:45:32.014485 (+ 3435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:45:32.014489 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:45:32.014568 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:45:32.014588 (+ 20us) client_negotiation.cc:769] Sending connection context
0925 02:45:32.014632 (+ 44us) client_negotiation.cc:240] Negotiation successful
0925 02:45:32.014689 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1767,"thread_start_us":95,"threads_started":1}
I20260925 02:45:32.014681 27394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:32.001035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.131:40941 (local address 127.0.0.1:46194)
0925 02:45:32.004946 (+ 3911us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:32.004959 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 02:45:32.005042 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:45:32.006728 (+ 1686us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:45:32.006731 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:45:32.006749 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:45:32.006838 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:45:32.006844 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:45:32.012056 (+ 5212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:45:32.012059 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:45:32.013308 (+ 1249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:45:32.013318 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:45:32.014279 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:45:32.014283 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:45:32.014371 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:45:32.014392 (+ 21us) client_negotiation.cc:769] Sending connection context
0925 02:45:32.014463 (+ 71us) client_negotiation.cc:240] Negotiation successful
0925 02:45:32.014531 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3848,"thread_start_us":96,"threads_started":1}
I20260925 02:45:32.015477 27395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:32.000903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59250 (local address 127.13.157.130:37541)
0925 02:45:32.005989 (+ 5086us) server_negotiation.cc:206] Beginning negotiation
0925 02:45:32.005995 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 02:45:32.006016 (+ 21us) server_negotiation.cc:407] Connection header received
0925 02:45:32.006128 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:45:32.006155 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:45:32.006349 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:45:32.007470 (+ 1121us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:45:32.007595 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:32.008598 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:32.011184 (+ 2586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:32.011932 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:32.014784 (+ 2852us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:45:32.015038 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0925 02:45:32.015227 (+ 189us) server_negotiation.cc:299] Negotiation successful
0925 02:45:32.015340 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4982,"thread_start_us":104,"threads_started":1}
I20260925 02:45:32.015492 27397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:32.001631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46194 (local address 127.13.157.131:40941)
0925 02:45:32.005636 (+ 4005us) server_negotiation.cc:206] Beginning negotiation
0925 02:45:32.005642 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 02:45:32.005663 (+ 21us) server_negotiation.cc:407] Connection header received
0925 02:45:32.005727 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:45:32.005731 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:45:32.006588 (+ 857us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:45:32.007054 (+ 466us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:45:32.008749 (+ 1695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:32.009829 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:32.013449 (+ 3620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:32.014152 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:32.014268 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:45:32.014400 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0925 02:45:32.015287 (+ 887us) server_negotiation.cc:299] Negotiation successful
0925 02:45:32.015378 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3895,"thread_start_us":144,"threads_started":1}
I20260925 02:45:32.016320 27033 tablet_service.cc:1467] Processing CreateTablet for tablet 48998434736247d39e8451ed84fb6ea2 (DEFAULT_TABLE table=test-table [id=a45e32678b294e6ab3ff322baceff0cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:45:32.017459 27033 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 48998434736247d39e8451ed84fb6ea2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:45:32.018164 27308 tablet_service.cc:1467] Processing CreateTablet for tablet 48998434736247d39e8451ed84fb6ea2 (DEFAULT_TABLE table=test-table [id=a45e32678b294e6ab3ff322baceff0cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:45:32.019012 27308 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 48998434736247d39e8451ed84fb6ea2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:45:32.021613 27159 tablet_service.cc:1467] Processing CreateTablet for tablet 48998434736247d39e8451ed84fb6ea2 (DEFAULT_TABLE table=test-table [id=a45e32678b294e6ab3ff322baceff0cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:45:32.022624 27159 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 48998434736247d39e8451ed84fb6ea2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:45:32.027032 27400 tablet_bootstrap.cc:492] T 48998434736247d39e8451ed84fb6ea2 P 1bab1df9399a4603be7fec4253ce9b58: Bootstrap starting.
I20260925 02:45:32.029322 27400 tablet_bootstrap.cc:654] T 48998434736247d39e8451ed84fb6ea2 P 1bab1df9399a4603be7fec4253ce9b58: Neither blocks nor log segments found. Creating new log.
I20260925 02:45:32.030117 27400 log.cc:824] T 48998434736247d39e8451ed84fb6ea2 P 1bab1df9399a4603be7fec4253ce9b58: Log is configured to *not* fsync() on all Append() calls
I20260925 02:45:32.030978 27401 tablet_bootstrap.cc:492] T 48998434736247d39e8451ed84fb6ea2 P 8a0747e35bdc45f5972ca95a228aad98: Bootstrap starting.
I20260925 02:45:32.033147 27401 tablet_bootstrap.cc:654] T 48998434736247d39e8451ed84fb6ea2 P 8a0747e35bdc45f5972ca95a228aad98: Neither blocks nor log segments found. Creating new log.
I20260925 02:45:32.034031 27401 log.cc:824] T 48998434736247d39e8451ed84fb6ea2 P 8a0747e35bdc45f5972ca95a228aad98: Log is configured to *not* fsync() on all Append() calls
I20260925 02:45:32.034694 27402 tablet_bootstrap.cc:492] T 48998434736247d39e8451ed84fb6ea2 P 22006d4c201a4713bce30c888fe40b5b: Bootstrap starting.
I20260925 02:45:32.036839 27402 tablet_bootstrap.cc:654] T 48998434736247d39e8451ed84fb6ea2 P 22006d4c201a4713bce30c888fe40b5b: Neither blocks nor log segments found. Creating new log.
I20260925 02:45:32.037694 27402 log.cc:824] T 48998434736247d39e8451ed84fb6ea2 P 22006d4c201a4713bce30c888fe40b5b: Log is configured to *not* fsync() on all Append() calls
I20260925 02:45:32.040237 27400 tablet_bootstrap.cc:492] T 48998434736247d39e8451ed84fb6ea2 P 1bab1df9399a4603be7fec4253ce9b58: No bootstrap required, opened a new log
I20260925 02:45:32.040555 27401 tablet_bootstrap.cc:492] T 48998434736247d39e8451ed84fb6ea2 P 8a0747e35bdc45f5972ca95a228aad98: No bootstrap required, opened a new log
I20260925 02:45:32.040802 27401 ts_tablet_manager.cc:1397] T 48998434736247d39e8451ed84fb6ea2 P 8a0747e35bdc45f5972ca95a228aad98: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.000s
I20260925 02:45:32.040884 27402 tablet_bootstrap.cc:492] T 48998434736247d39e8451ed84fb6ea2 P 22006d4c201a4713bce30c888fe40b5b: No bootstrap required, opened a new log
I20260925 02:45:32.041061 27400 ts_tablet_manager.cc:1397] T 48998434736247d39e8451ed84fb6ea2 P 1bab1df9399a4603be7fec4253ce9b58: Time spent bootstrapping tablet: real 0.014s user 0.005s sys 0.000s
I20260925 02:45:32.041077 27402 ts_tablet_manager.cc:1397] T 48998434736247d39e8451ed84fb6ea2 P 22006d4c201a4713bce30c888fe40b5b: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260925 02:45:32.045118 27402 raft_consensus.cc:348] T 48998434736247d39e8451ed84fb6ea2 P 22006d4c201a4713bce30c888fe40b5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22006d4c201a4713bce30c888fe40b5b" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 37541 } } peers { permanent_uuid: "8a0747e35bdc45f5972ca95a228aad98" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 40941 } } peers { permanent_uuid: "1bab1df9399a4603be7fec4253ce9b58" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 34073 } }
I20260925 02:45:32.045428 27402 raft_consensus.cc:374] T 48998434736247d39e8451ed84fb6ea2 P 22006d4c201a4713bce30c888fe40b5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:45:32.045497 27402 raft_consensus.cc:729] T 48998434736247d39e8451ed84fb6ea2 P 22006d4c201a4713bce30c888fe40b5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22006d4c201a4713bce30c888fe40b5b, State: Initialized, Role: FOLLOWER
I20260925 02:45:32.046030 27402 consensus_queue.cc:260] T 48998434736247d39e8451ed84fb6ea2 P 22006d4c201a4713bce30c888fe40b5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22006d4c201a4713bce30c888fe40b5b" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 37541 } } peers { permanent_uuid: "8a0747e35bdc45f5972ca95a228aad98" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 40941 } } peers { permanent_uuid: "1bab1df9399a4603be7fec4253ce9b58" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 34073 } }
I20260925 02:45:32.047165 27402 ts_tablet_manager.cc:1428] T 48998434736247d39e8451ed84fb6ea2 P 22006d4c201a4713bce30c888fe40b5b: Time spent starting tablet: real 0.006s user 0.006s sys 0.001s
I20260925 02:45:32.047528 27234 heartbeater.cc:503] Master 127.13.157.190:40793 was elected leader, sending a full tablet report...
I20260925 02:45:32.049505 27401 raft_consensus.cc:348] T 48998434736247d39e8451ed84fb6ea2 P 8a0747e35bdc45f5972ca95a228aad98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22006d4c201a4713bce30c888fe40b5b" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 37541 } } peers { permanent_uuid: "8a0747e35bdc45f5972ca95a228aad98" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 40941 } } peers { permanent_uuid: "1bab1df9399a4603be7fec4253ce9b58" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 34073 } }
I20260925 02:45:32.051183 27401 raft_consensus.cc:374] T 48998434736247d39e8451ed84fb6ea2 P 8a0747e35bdc45f5972ca95a228aad98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:45:32.051258 27401 raft_consensus.cc:729] T 48998434736247d39e8451ed84fb6ea2 P 8a0747e35bdc45f5972ca95a228aad98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a0747e35bdc45f5972ca95a228aad98, State: Initialized, Role: FOLLOWER
I20260925 02:45:32.051231 27400 raft_consensus.cc:348] T 48998434736247d39e8451ed84fb6ea2 P 1bab1df9399a4603be7fec4253ce9b58 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22006d4c201a4713bce30c888fe40b5b" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 37541 } } peers { permanent_uuid: "8a0747e35bdc45f5972ca95a228aad98" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 40941 } } peers { permanent_uuid: "1bab1df9399a4603be7fec4253ce9b58" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 34073 } }
I20260925 02:45:32.051649 27400 raft_consensus.cc:374] T 48998434736247d39e8451ed84fb6ea2 P 1bab1df9399a4603be7fec4253ce9b58 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:45:32.051745 27400 raft_consensus.cc:729] T 48998434736247d39e8451ed84fb6ea2 P 1bab1df9399a4603be7fec4253ce9b58 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bab1df9399a4603be7fec4253ce9b58, State: Initialized, Role: FOLLOWER
I20260925 02:45:32.051733 27401 consensus_queue.cc:260] T 48998434736247d39e8451ed84fb6ea2 P 8a0747e35bdc45f5972ca95a228aad98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22006d4c201a4713bce30c888fe40b5b" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 37541 } } peers { permanent_uuid: "8a0747e35bdc45f5972ca95a228aad98" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 40941 } } peers { permanent_uuid: "1bab1df9399a4603be7fec4253ce9b58" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 34073 } }
I20260925 02:45:32.052294 27400 consensus_queue.cc:260] T 48998434736247d39e8451ed84fb6ea2 P 1bab1df9399a4603be7fec4253ce9b58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22006d4c201a4713bce30c888fe40b5b" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 37541 } } peers { permanent_uuid: "8a0747e35bdc45f5972ca95a228aad98" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 40941 } } peers { permanent_uuid: "1bab1df9399a4603be7fec4253ce9b58" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 34073 } }
I20260925 02:45:32.052802 27401 ts_tablet_manager.cc:1428] T 48998434736247d39e8451ed84fb6ea2 P 8a0747e35bdc45f5972ca95a228aad98: Time spent starting tablet: real 0.009s user 0.006s sys 0.000s
I20260925 02:45:32.053156 27374 heartbeater.cc:503] Master 127.13.157.190:40793 was elected leader, sending a full tablet report...
I20260925 02:45:32.054385 27099 heartbeater.cc:503] Master 127.13.157.190:40793 was elected leader, sending a full tablet report...
I20260925 02:45:32.054687 27400 ts_tablet_manager.cc:1428] T 48998434736247d39e8451ed84fb6ea2 P 1bab1df9399a4603be7fec4253ce9b58: Time spent starting tablet: real 0.014s user 0.006s sys 0.000s
I20260925 02:45:32.080395 27406 raft_consensus.cc:482] T 48998434736247d39e8451ed84fb6ea2 P 22006d4c201a4713bce30c888fe40b5b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 02:45:32.080660 27406 raft_consensus.cc:504] T 48998434736247d39e8451ed84fb6ea2 P 22006d4c201a4713bce30c888fe40b5b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22006d4c201a4713bce30c888fe40b5b" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 37541 } } peers { permanent_uuid: "8a0747e35bdc45f5972ca95a228aad98" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 40941 } } peers { permanent_uuid: "1bab1df9399a4603be7fec4253ce9b58" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 34073 } }
I20260925 02:45:32.082247 27406 leader_election.cc:288] T 48998434736247d39e8451ed84fb6ea2 P 22006d4c201a4713bce30c888fe40b5b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a0747e35bdc45f5972ca95a228aad98 (127.13.157.131:40941), 1bab1df9399a4603be7fec4253ce9b58 (127.13.157.129:34073)
I20260925 02:45:32.087703 27397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:32.083416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.130:49207 (local address 127.13.157.131:40941)
0925 02:45:32.083539 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0925 02:45:32.083544 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:45:32.083558 (+ 14us) server_negotiation.cc:407] Connection header received
0925 02:45:32.083610 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:45:32.083613 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:45:32.083680 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:45:32.083755 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:45:32.084084 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:32.084890 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:32.086457 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:32.087002 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:32.087095 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:45:32.087149 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0925 02:45:32.087472 (+ 323us) server_negotiation.cc:299] Negotiation successful
0925 02:45:32.087554 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260925 02:45:32.088078 27399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:32.083450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.130:39393 (local address 127.13.157.129:34073)
0925 02:45:32.083566 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0925 02:45:32.083570 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:45:32.083690 (+ 120us) server_negotiation.cc:407] Connection header received
0925 02:45:32.083839 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:45:32.083842 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:45:32.083891 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:45:32.083937 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:45:32.084417 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:32.085157 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:32.086706 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:32.087351 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:32.087452 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:45:32.087508 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0925 02:45:32.087869 (+ 361us) server_negotiation.cc:299] Negotiation successful
0925 02:45:32.087951 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260925 02:45:32.088541 27410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:32.083068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.129:34073 (local address 127.13.157.130:39393)
0925 02:45:32.083602 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:32.083614 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 02:45:32.083708 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:45:32.084069 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:45:32.084072 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:45:32.084090 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:45:32.084274 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:45:32.084283 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:45:32.085305 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:45:32.085308 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:45:32.086549 (+ 1241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:45:32.086561 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:45:32.087609 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:45:32.087613 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:45:32.087718 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:45:32.087742 (+ 24us) client_negotiation.cc:769] Sending connection context
0925 02:45:32.088244 (+ 502us) client_negotiation.cc:240] Negotiation successful
0925 02:45:32.088349 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":478,"mutex_wait_us":2,"thread_start_us":103,"threads_started":1}
I20260925 02:45:32.088557 27409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:32.082632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.131:40941 (local address 127.13.157.130:49207)
0925 02:45:32.083080 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:32.083097 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 02:45:32.083220 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:45:32.083776 (+ 556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:45:32.083779 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:45:32.083803 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:45:32.083949 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:45:32.083954 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:45:32.085027 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:45:32.085030 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:45:32.086314 (+ 1284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:45:32.086324 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:45:32.087240 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:45:32.087244 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:45:32.087340 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:45:32.087364 (+ 24us) client_negotiation.cc:769] Sending connection context
0925 02:45:32.088087 (+ 723us) client_negotiation.cc:240] Negotiation successful
0925 02:45:32.088381 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":356,"mutex_wait_us":21,"thread_start_us":134,"threads_started":1}
I20260925 02:45:32.089329 27035 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "48998434736247d39e8451ed84fb6ea2" candidate_uuid: "22006d4c201a4713bce30c888fe40b5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bab1df9399a4603be7fec4253ce9b58" is_pre_election: true
I20260925 02:45:32.089696 27035 raft_consensus.cc:2454] T 48998434736247d39e8451ed84fb6ea2 P 1bab1df9399a4603be7fec4253ce9b58 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 22006d4c201a4713bce30c888fe40b5b in term 0.
I20260925 02:45:32.090245 27122 leader_election.cc:302] T 48998434736247d39e8451ed84fb6ea2 P 22006d4c201a4713bce30c888fe40b5b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bab1df9399a4603be7fec4253ce9b58, 22006d4c201a4713bce30c888fe40b5b; no voters:
I20260925 02:45:32.090294 27328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "48998434736247d39e8451ed84fb6ea2" candidate_uuid: "22006d4c201a4713bce30c888fe40b5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a0747e35bdc45f5972ca95a228aad98" is_pre_election: true
I20260925 02:45:32.090548 27328 raft_consensus.cc:2454] T 48998434736247d39e8451ed84fb6ea2 P 8a0747e35bdc45f5972ca95a228aad98 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 22006d4c201a4713bce30c888fe40b5b in term 0.
I20260925 02:45:32.090577 27406 raft_consensus.cc:2781] T 48998434736247d39e8451ed84fb6ea2 P 22006d4c201a4713bce30c888fe40b5b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 02:45:32.090658 27406 raft_consensus.cc:482] T 48998434736247d39e8451ed84fb6ea2 P 22006d4c201a4713bce30c888fe40b5b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 02:45:32.090699 27406 raft_consensus.cc:3037] T 48998434736247d39e8451ed84fb6ea2 P 22006d4c201a4713bce30c888fe40b5b [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:45:32.091897 27406 raft_consensus.cc:504] T 48998434736247d39e8451ed84fb6ea2 P 22006d4c201a4713bce30c888fe40b5b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22006d4c201a4713bce30c888fe40b5b" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 37541 } } peers { permanent_uuid: "8a0747e35bdc45f5972ca95a228aad98" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 40941 } } peers { permanent_uuid: "1bab1df9399a4603be7fec4253ce9b58" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 34073 } }
I20260925 02:45:32.092247 27406 leader_election.cc:288] T 48998434736247d39e8451ed84fb6ea2 P 22006d4c201a4713bce30c888fe40b5b [CANDIDATE]: Term 1 election: Requested vote from peers 8a0747e35bdc45f5972ca95a228aad98 (127.13.157.131:40941), 1bab1df9399a4603be7fec4253ce9b58 (127.13.157.129:34073)
I20260925 02:45:32.092588 27328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "48998434736247d39e8451ed84fb6ea2" candidate_uuid: "22006d4c201a4713bce30c888fe40b5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a0747e35bdc45f5972ca95a228aad98"
I20260925 02:45:32.092707 27328 raft_consensus.cc:3037] T 48998434736247d39e8451ed84fb6ea2 P 8a0747e35bdc45f5972ca95a228aad98 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:45:32.093739 27328 raft_consensus.cc:2454] T 48998434736247d39e8451ed84fb6ea2 P 8a0747e35bdc45f5972ca95a228aad98 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 22006d4c201a4713bce30c888fe40b5b in term 1.
I20260925 02:45:32.094020 27120 leader_election.cc:302] T 48998434736247d39e8451ed84fb6ea2 P 22006d4c201a4713bce30c888fe40b5b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22006d4c201a4713bce30c888fe40b5b, 8a0747e35bdc45f5972ca95a228aad98; no voters:
I20260925 02:45:32.094096 27035 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "48998434736247d39e8451ed84fb6ea2" candidate_uuid: "22006d4c201a4713bce30c888fe40b5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bab1df9399a4603be7fec4253ce9b58"
I20260925 02:45:32.094259 27406 raft_consensus.cc:2781] T 48998434736247d39e8451ed84fb6ea2 P 22006d4c201a4713bce30c888fe40b5b [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:45:32.094228 27035 raft_consensus.cc:3037] T 48998434736247d39e8451ed84fb6ea2 P 1bab1df9399a4603be7fec4253ce9b58 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:45:32.094475 27406 raft_consensus.cc:686] T 48998434736247d39e8451ed84fb6ea2 P 22006d4c201a4713bce30c888fe40b5b [term 1 LEADER]: Becoming Leader. State: Replica: 22006d4c201a4713bce30c888fe40b5b, State: Running, Role: LEADER
I20260925 02:45:32.094861 27406 consensus_queue.cc:237] T 48998434736247d39e8451ed84fb6ea2 P 22006d4c201a4713bce30c888fe40b5b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22006d4c201a4713bce30c888fe40b5b" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 37541 } } peers { permanent_uuid: "8a0747e35bdc45f5972ca95a228aad98" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 40941 } } peers { permanent_uuid: "1bab1df9399a4603be7fec4253ce9b58" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 34073 } }
I20260925 02:45:32.095501 27035 raft_consensus.cc:2454] T 48998434736247d39e8451ed84fb6ea2 P 1bab1df9399a4603be7fec4253ce9b58 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 22006d4c201a4713bce30c888fe40b5b in term 1.
I20260925 02:45:32.098672 26901 catalog_manager.cc:5495] T 48998434736247d39e8451ed84fb6ea2 P 22006d4c201a4713bce30c888fe40b5b reported cstate change: term changed from 0 to 1, leader changed from <none> to 22006d4c201a4713bce30c888fe40b5b (127.13.157.130). New cstate: current_term: 1 leader_uuid: "22006d4c201a4713bce30c888fe40b5b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22006d4c201a4713bce30c888fe40b5b" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 37541 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a0747e35bdc45f5972ca95a228aad98" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 40941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1bab1df9399a4603be7fec4253ce9b58" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 34073 } health_report { overall_health: UNKNOWN } } }
I20260925 02:45:32.110175 27395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:32.106434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59266 (local address 127.13.157.130:37541)
0925 02:45:32.106550 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0925 02:45:32.106554 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:45:32.106612 (+ 58us) server_negotiation.cc:407] Connection header received
0925 02:45:32.106898 (+ 286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:45:32.106925 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:45:32.106989 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:45:32.107078 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 02:45:32.107335 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:32.107902 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:32.108747 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:32.108969 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:32.109053 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:45:32.109349 (+ 296us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 02:45:32.109770 (+ 421us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 02:45:32.109880 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0925 02:45:32.109942 (+ 62us) server_negotiation.cc:299] Negotiation successful
0925 02:45:32.109999 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260925 02:45:32.118657 27035 raft_consensus.cc:1261] T 48998434736247d39e8451ed84fb6ea2 P 1bab1df9399a4603be7fec4253ce9b58 [term 1 FOLLOWER]: Refusing update from remote peer 22006d4c201a4713bce30c888fe40b5b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:45:32.118671 27328 raft_consensus.cc:1261] T 48998434736247d39e8451ed84fb6ea2 P 8a0747e35bdc45f5972ca95a228aad98 [term 1 FOLLOWER]: Refusing update from remote peer 22006d4c201a4713bce30c888fe40b5b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:45:32.119488 27406 consensus_queue.cc:1035] T 48998434736247d39e8451ed84fb6ea2 P 22006d4c201a4713bce30c888fe40b5b [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a0747e35bdc45f5972ca95a228aad98" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 40941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:45:32.119890 27406 consensus_queue.cc:1035] T 48998434736247d39e8451ed84fb6ea2 P 22006d4c201a4713bce30c888fe40b5b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1bab1df9399a4603be7fec4253ce9b58" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 34073 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:45:32.129195 27415 mvcc.cc:205] Tried to move back new op lower bound from 7333086544345616384 to 7333086544264409088. Current Snapshot: MvccSnapshot[applied={T|T < 7333086544345616384}]
I20260925 02:45:32.131771 27417 mvcc.cc:205] Tried to move back new op lower bound from 7333086544345616384 to 7333086544264409088. Current Snapshot: MvccSnapshot[applied={T|T < 7333086544345616384}]
I20260925 02:45:32.138502 27416 mvcc.cc:205] Tried to move back new op lower bound from 7333086544345616384 to 7333086544264409088. Current Snapshot: MvccSnapshot[applied={T|T < 7333086544345616384}]
I20260925 02:45:32.141090 26900 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39562:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 02:45:32.141268 26900 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39562:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 02:45:32.143913 26900 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 28497343617d4cf396f60d556f7be937: Sending DeleteTablet for 3 replicas of tablet 48998434736247d39e8451ed84fb6ea2
I20260925 02:45:32.145390 27159 tablet_service.cc:1514] Processing DeleteTablet for tablet 48998434736247d39e8451ed84fb6ea2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:45:32 UTC) from {username='kudu', principal='kudu/127.13.157.190@KRBTEST.COM'} at 127.0.0.1:59250
I20260925 02:45:32.145890 27424 tablet_replica.cc:330] T 48998434736247d39e8451ed84fb6ea2 P 22006d4c201a4713bce30c888fe40b5b: stopping tablet replica
I20260925 02:45:32.145920 27308 tablet_service.cc:1514] Processing DeleteTablet for tablet 48998434736247d39e8451ed84fb6ea2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:45:32 UTC) from {username='kudu', principal='kudu/127.13.157.190@KRBTEST.COM'} at 127.0.0.1:46194
I20260925 02:45:32.146364 27424 raft_consensus.cc:2229] T 48998434736247d39e8451ed84fb6ea2 P 22006d4c201a4713bce30c888fe40b5b [term 1 LEADER]: Raft consensus shutting down.
I20260925 02:45:32.146385 27033 tablet_service.cc:1514] Processing DeleteTablet for tablet 48998434736247d39e8451ed84fb6ea2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:45:32 UTC) from {username='kudu', principal='kudu/127.13.157.190@KRBTEST.COM'} at 127.0.0.1:52072
I20260925 02:45:32.146934 27424 raft_consensus.cc:2258] T 48998434736247d39e8451ed84fb6ea2 P 22006d4c201a4713bce30c888fe40b5b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:45:32.147696 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 26939
I20260925 02:45:32.149416 27424 ts_tablet_manager.cc:1905] T 48998434736247d39e8451ed84fb6ea2 P 22006d4c201a4713bce30c888fe40b5b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:45:32.150012 27425 tablet_replica.cc:330] T 48998434736247d39e8451ed84fb6ea2 P 8a0747e35bdc45f5972ca95a228aad98: stopping tablet replica
I20260925 02:45:32.150342 27425 raft_consensus.cc:2229] T 48998434736247d39e8451ed84fb6ea2 P 8a0747e35bdc45f5972ca95a228aad98 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 02:45:32.150619 27425 raft_consensus.cc:2258] T 48998434736247d39e8451ed84fb6ea2 P 8a0747e35bdc45f5972ca95a228aad98 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:45:32.152626 27424 ts_tablet_manager.cc:1918] T 48998434736247d39e8451ed84fb6ea2 P 22006d4c201a4713bce30c888fe40b5b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 02:45:32.152709 27424 log.cc:1196] T 48998434736247d39e8451ed84fb6ea2 P 22006d4c201a4713bce30c888fe40b5b: Deleting WAL directory at /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790304178325258-13942-0/minicluster-data/ts-1/wal/wals/48998434736247d39e8451ed84fb6ea2
I20260925 02:45:32.152961 27424 ts_tablet_manager.cc:1939] T 48998434736247d39e8451ed84fb6ea2 P 22006d4c201a4713bce30c888fe40b5b: Deleting consensus metadata
I20260925 02:45:32.154039 26886 catalog_manager.cc:4841] TS 22006d4c201a4713bce30c888fe40b5b (127.13.157.130:37541): tablet 48998434736247d39e8451ed84fb6ea2 (table test-table [id=a45e32678b294e6ab3ff322baceff0cf]) successfully deleted
W20260925 02:45:32.156149 26887 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.157.129:34073 (error 108)
I20260925 02:45:32.157006 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 27103
I20260925 02:45:32.161123 27394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:32.160758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.129:34073 (local address 127.0.0.1:52086)
0925 02:45:32.160955 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:32.161028 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.129:34073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116}
W20260925 02:45:32.161324 26887 catalog_manager.cc:4584] TS 1bab1df9399a4603be7fec4253ce9b58 (127.13.157.129:34073): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 48998434736247d39e8451ed84fb6ea2: Network error: Client connection negotiation failed: client connection to 127.13.157.129:34073: connect: Connection refused (error 111)
I20260925 02:45:32.163659 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 27237
I20260925 02:45:32.171191 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 26868
I20260925 02:45:32.177487 13942 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260925 02:45:32.512774 13942 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260925 02:45:32.512890 13942 mini_ranger.cc:66] Stopping Ranger...
I20260925 02:45:32.847713 13942 mini_ranger.cc:68] Stopped Ranger
2026-09-25 02:45:32.847 UTC [25926] LOG: received smart shutdown request
2026-09-25 02:45:32.852 UTC [25926] LOG: background worker "logical replication launcher" (PID 25935) exited with exit code 1
2026-09-25 02:45:32.852 UTC [25930] LOG: shutting down
2026-09-25 02:45:33.025 UTC [25926] LOG: database system is shut down
2026-09-25T02:45:33Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (46685 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790304178325258-13942-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 02:45:33 dist-test-slave-kznw krb5kdc[27436](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 02:45:33 dist-test-slave-kznw krb5kdc[27436](info): set up 2 sockets
Sep 25 02:45:33 dist-test-slave-kznw krb5kdc[27436](info): commencing operation
krb5kdc: starting...
W20260925 02:45:34.078439 13942 mini_kdc.cc:121] Time spent starting KDC: real 0.933s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790304178325258-13942-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790304178325258-13942-0/krb5kdc/test-user.keytab.
Sep 25 02:45:34 dist-test-slave-kznw krb5kdc[27436](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304334, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T02:45:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T02:45:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.157.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.157.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.157.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab.
Entry for principal kudu/127.13.157.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab.
I20260925 02:45:34.197537 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790304178325258-13942-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790304178325258-13942-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790304178325258-13942-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.157.190:36279
--webserver_interface=127.13.157.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab
--principal=kudu/127.13.157.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:35407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.157.190:36279,127.13.157.189:46813,127.13.157.188:41595
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:45:34.309233 27449 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:45:34.309521 27449 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:45:34.309579 27449 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:45:34.313731 27449 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:45:34.313810 27449 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:45:34.313838 27449 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:45:34.313860 27449 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:45:34.313880 27449 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:45:34.319489 27449 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:35407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790304178325258-13942-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790304178325258-13942-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.157.190:36279,127.13.157.189:46813,127.13.157.188:41595
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab
--principal=kudu/127.13.157.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.157.190:36279
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.157.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27449
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790304178325258-13942-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:45:34.320654 27449 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:45:34.321720 27449 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:45:34.330505 27455 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:45:34.330623 27449 server_base.cc:1034] running on GCE node
W20260925 02:45:34.330473 27454 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:45:34.330508 27457 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:45:34.331518 27449 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:45:34.332641 27449 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:45:34.333801 27449 hybrid_clock.cc:648] HybridClock initialized: now 1790304334333771 us; error 41 us; skew 500 ppm
Sep 25 02:45:34 dist-test-slave-kznw krb5kdc[27436](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304334, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:45:34.342015 27449 init.cc:377] Logged in from keytab as kudu/127.13.157.190@KRBTEST.COM (short username kudu)
I20260925 02:45:34.343043 27449 webserver.cc:460] Webserver started at http://127.13.157.190:41397/ using document root <none> and password file <none>
I20260925 02:45:34.343617 27449 fs_manager.cc:359] Metadata directory not provided
I20260925 02:45:34.343685 27449 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:45:34.343863 27449 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:45:34.345966 27449 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790304178325258-13942-0/minicluster-data/master-0/data/instance:
uuid: "7f34aeb99aad45a2a58463a851771b79"
format_stamp: "Formatted at 2026-09-25 02:45:34 on dist-test-slave-kznw"
server_key: "9e805f9295173d3e0d4ed3dcfaf59505"
server_key_iv: "9ec092ccfb689ad2b574f46b910b1e14"
server_key_version: "encryptionkey@0"
I20260925 02:45:34.346681 27449 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790304178325258-13942-0/minicluster-data/master-0/wal/instance:
uuid: "7f34aeb99aad45a2a58463a851771b79"
format_stamp: "Formatted at 2026-09-25 02:45:34 on dist-test-slave-kznw"
server_key: "9e805f9295173d3e0d4ed3dcfaf59505"
server_key_iv: "9ec092ccfb689ad2b574f46b910b1e14"
server_key_version: "encryptionkey@0"
I20260925 02:45:34.350468 27449 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260925 02:45:34.352859 27464 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:45:34.354161 27449 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260925 02:45:34.354277 27449 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790304178325258-13942-0/minicluster-data/master-0/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790304178325258-13942-0/minicluster-data/master-0/wal
uuid: "7f34aeb99aad45a2a58463a851771b79"
format_stamp: "Formatted at 2026-09-25 02:45:34 on dist-test-slave-kznw"
server_key: "9e805f9295173d3e0d4ed3dcfaf59505"
server_key_iv: "9ec092ccfb689ad2b574f46b910b1e14"
server_key_version: "encryptionkey@0"
I20260925 02:45:34.354367 27449 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790304178325258-13942-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790304178325258-13942-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790304178325258-13942-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:45:34.370761 27449 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:45:34.373932 27449 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:45:34.374086 27449 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:45:34.381170 27449 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.190:36279
I20260925 02:45:34.381443 27516 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.190:36279 every 8 connection(s)
I20260925 02:45:34.382184 27449 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
I20260925 02:45:34.383917 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 27449
I20260925 02:45:34.384027 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790304178325258-13942-0/minicluster-data/master-0/wal/instance
I20260925 02:45:34.384312 13942 external_mini_cluster.cc:1428] Setting key b4aa75b8bf3d17142764f9f6d0dfbf2f
I20260925 02:45:34.385134 27517 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:45:34.389281 27517 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.157.190" port: 36279 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.13.157.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.157.189@KRBTEST.COM" created.
Sep 25 02:45:34 dist-test-slave-kznw krb5kdc[27436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304334, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.190@KRBTEST.COM for kudu/127.13.157.190@KRBTEST.COM
I20260925 02:45:34.404192 27521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:34.391611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:36279 (local address 127.0.0.1:39602)
0925 02:45:34.392186 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:34.392256 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0925 02:45:34.393088 (+ 832us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:45:34.393854 (+ 766us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:45:34.393860 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:45:34.394283 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:45:34.394620 (+ 337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:45:34.394639 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:45:34.395896 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:45:34.395899 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:45:34.396885 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:45:34.396895 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:45:34.397513 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:45:34.397517 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:45:34.397621 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:45:34.398166 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:45:34.398188 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:45:34.399801 (+ 1613us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:45:34.401528 (+ 1727us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:45:34.401536 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:45:34.401546 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:45:34.401756 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:45:34.402040 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:45:34.402043 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:45:34.402045 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:45:34.402146 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:45:34.402698 (+ 552us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:45:34.402704 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:45:34.402840 (+ 136us) client_negotiation.cc:769] Sending connection context
0925 02:45:34.403028 (+ 188us) client_negotiation.cc:240] Negotiation successful
0925 02:45:34.403396 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":394,"mutex_wait_us":101,"spinlock_wait_cycles":48896,"thread_start_us":164,"threads_started":1}
I20260925 02:45:34.404201 27520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:34.391611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39602 (local address 127.13.157.190:36279)
0925 02:45:34.392088 (+ 477us) server_negotiation.cc:206] Beginning negotiation
0925 02:45:34.392116 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0925 02:45:34.392362 (+ 246us) server_negotiation.cc:407] Connection header received
0925 02:45:34.393422 (+ 1060us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:45:34.393428 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:45:34.393615 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:45:34.393781 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:45:34.394792 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:34.395766 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:34.396993 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:34.397360 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:34.397503 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:45:34.399932 (+ 2429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:45:34.399938 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:45:34.399947 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:45:34.399968 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:45:34.401430 (+ 1462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:45:34.401854 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:45:34.401859 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:45:34.401860 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:45:34.401959 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:45:34.402235 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:45:34.402238 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:45:34.402240 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:45:34.402593 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:45:34.402717 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0925 02:45:34.403135 (+ 418us) server_negotiation.cc:299] Negotiation successful
0925 02:45:34.403334 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":317,"thread_start_us":148,"threads_started":1}
I20260925 02:45:34.406489 27517 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.157.189" port: 46813 } has no permanent_uuid. Determining permanent_uuid...
I20260925 02:45:34.407285 27521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:34.406984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.189:46813 (local address 127.0.0.1:39452)
0925 02:45:34.407142 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:34.407222 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.189:46813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
W20260925 02:45:34.407490 27468 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.157.189:46813: connect: Connection refused (error 111)
I20260925 02:45:34.408464 27521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:34.408209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.189:46813 (local address 127.0.0.1:39460)
0925 02:45:34.408350 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:34.408407 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.189:46813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260925 02:45:34.408614 27517 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.157.189:46813: Network error: Client connection negotiation failed: client connection to 127.13.157.189:46813: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.157.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.189.keytab.
Entry for principal kudu/127.13.157.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.189.keytab.
I20260925 02:45:34.410262 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790304178325258-13942-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790304178325258-13942-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790304178325258-13942-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790304178325258-13942-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.157.189:46813
--webserver_interface=127.13.157.189
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.189.keytab
--principal=kudu/127.13.157.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:35407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.157.190:36279,127.13.157.189:46813,127.13.157.188:41595
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 02:45:34.457855 27517 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.157.189" port: 46813 } attempt: 1
I20260925 02:45:34.458866 27521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:34.458471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.189:46813 (local address 127.0.0.1:39468)
0925 02:45:34.458691 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:34.458761 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.189:46813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260925 02:45:34.459589 27521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:34.459374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.189:46813 (local address 127.0.0.1:39474)
0925 02:45:34.459481 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:34.459529 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.189:46813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260925 02:45:34.459710 27517 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.157.189:46813: Network error: Client connection negotiation failed: client connection to 127.13.157.189:46813: connect: Connection refused (error 111)
W20260925 02:45:34.519212 27524 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:45:34.519493 27524 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:45:34.519534 27524 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:45:34.523478 27524 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:45:34.523530 27524 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:45:34.523547 27524 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:45:34.523562 27524 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:45:34.523577 27524 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:45:34.527868 27517 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.157.189" port: 46813 } attempt: 2
I20260925 02:45:34.528612 27521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:34.528337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.189:46813 (local address 127.0.0.1:39484)
0925 02:45:34.528477 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:34.528539 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.189:46813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260925 02:45:34.528806 27524 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:35407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790304178325258-13942-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790304178325258-13942-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.13.157.190:36279,127.13.157.189:46813,127.13.157.188:41595
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.189.keytab
--principal=kudu/127.13.157.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.157.189:46813
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790304178325258-13942-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.13.157.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27524
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790304178325258-13942-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:45:34.529415 27521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:34.529142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.189:46813 (local address 127.0.0.1:39496)
0925 02:45:34.529285 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:34.529346 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.189:46813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260925 02:45:34.529547 27517 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.157.189:46813: Network error: Client connection negotiation failed: client connection to 127.13.157.189:46813: connect: Connection refused (error 111)
I20260925 02:45:34.529975 27524 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:45:34.531008 27524 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:45:34.539099 27529 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:45:34.539099 27530 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:45:34.539099 27532 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:45:34.539153 27524 server_base.cc:1034] running on GCE node
I20260925 02:45:34.539876 27524 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:45:34.540925 27524 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:45:34.542084 27524 hybrid_clock.cc:648] HybridClock initialized: now 1790304334542066 us; error 34 us; skew 500 ppm
Sep 25 02:45:34 dist-test-slave-kznw krb5kdc[27436](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304334, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:45:34.544749 27524 init.cc:377] Logged in from keytab as kudu/127.13.157.189@KRBTEST.COM (short username kudu)
I20260925 02:45:34.545604 27524 webserver.cc:460] Webserver started at http://127.13.157.189:36423/ using document root <none> and password file <none>
I20260925 02:45:34.546087 27524 fs_manager.cc:359] Metadata directory not provided
I20260925 02:45:34.546141 27524 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:45:34.546305 27524 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:45:34.548292 27524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790304178325258-13942-0/minicluster-data/master-1/data/instance:
uuid: "894f7d93b7df4cd19c802e234a6c761b"
format_stamp: "Formatted at 2026-09-25 02:45:34 on dist-test-slave-kznw"
server_key: "ac9e5b10274bdd9067f30653b2f91c13"
server_key_iv: "60cc86046c24a68ba3e89d6bd27642cd"
server_key_version: "encryptionkey@0"
I20260925 02:45:34.548789 27524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790304178325258-13942-0/minicluster-data/master-1/wal/instance:
uuid: "894f7d93b7df4cd19c802e234a6c761b"
format_stamp: "Formatted at 2026-09-25 02:45:34 on dist-test-slave-kznw"
server_key: "ac9e5b10274bdd9067f30653b2f91c13"
server_key_iv: "60cc86046c24a68ba3e89d6bd27642cd"
server_key_version: "encryptionkey@0"
I20260925 02:45:34.552222 27524 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260925 02:45:34.554436 27539 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:45:34.555647 27524 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260925 02:45:34.555763 27524 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790304178325258-13942-0/minicluster-data/master-1/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790304178325258-13942-0/minicluster-data/master-1/wal
uuid: "894f7d93b7df4cd19c802e234a6c761b"
format_stamp: "Formatted at 2026-09-25 02:45:34 on dist-test-slave-kznw"
server_key: "ac9e5b10274bdd9067f30653b2f91c13"
server_key_iv: "60cc86046c24a68ba3e89d6bd27642cd"
server_key_version: "encryptionkey@0"
I20260925 02:45:34.555855 27524 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790304178325258-13942-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790304178325258-13942-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790304178325258-13942-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:45:34.587096 27524 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:45:34.590384 27524 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:45:34.590571 27524 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:45:34.597671 27524 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.189:46813
I20260925 02:45:34.597920 27591 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.189:46813 every 8 connection(s)
I20260925 02:45:34.598639 27524 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790304178325258-13942-0/minicluster-data/master-1/data/info.pb
I20260925 02:45:34.601418 27592 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:45:34.605139 27592 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.157.190" port: 36279 } has no permanent_uuid. Determining permanent_uuid...
I20260925 02:45:34.606493 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 27524
I20260925 02:45:34.606660 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790304178325258-13942-0/minicluster-data/master-1/wal/instance
I20260925 02:45:34.606971 13942 external_mini_cluster.cc:1428] Setting key 86b4713a0d61f7ba4dd92c7998d33639
WARNING: no policy specified for kudu/127.13.157.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.157.188@KRBTEST.COM" created.
Sep 25 02:45:34 dist-test-slave-kznw krb5kdc[27436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304334, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.189@KRBTEST.COM for kudu/127.13.157.190@KRBTEST.COM
I20260925 02:45:34.620695 27517 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.157.189" port: 46813 } attempt: 3
I20260925 02:45:34.621636 27520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:34.607018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39604 (local address 127.13.157.190:36279)
0925 02:45:34.607180 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0925 02:45:34.607184 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:45:34.608058 (+ 874us) server_negotiation.cc:407] Connection header received
0925 02:45:34.609018 (+ 960us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:45:34.609021 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:45:34.609079 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:45:34.609153 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:45:34.610327 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:34.610847 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:34.611961 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:34.612199 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:34.612296 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:45:34.617476 (+ 5180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:45:34.617480 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:45:34.617483 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:45:34.617510 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:45:34.619087 (+ 1577us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:45:34.619713 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:45:34.619716 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:45:34.619717 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:45:34.619799 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:45:34.620134 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:45:34.620136 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:45:34.620137 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:45:34.620327 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:45:34.620399 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0925 02:45:34.621127 (+ 728us) server_negotiation.cc:299] Negotiation successful
0925 02:45:34.621480 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260925 02:45:34.622527 27594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:34.607412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:36279 (local address 127.0.0.1:39604)
0925 02:45:34.607884 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:34.607937 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0925 02:45:34.608804 (+ 867us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:45:34.609332 (+ 528us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:45:34.609339 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:45:34.609688 (+ 349us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:45:34.610174 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:45:34.610188 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:45:34.610983 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:45:34.610986 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:45:34.611838 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:45:34.611848 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:45:34.612339 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:45:34.612343 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:45:34.612504 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:45:34.613100 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:45:34.613116 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:45:34.617305 (+ 4189us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:45:34.619206 (+ 1901us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:45:34.619214 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:45:34.619230 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:45:34.619599 (+ 369us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:45:34.619878 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:45:34.619881 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:45:34.619883 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:45:34.620057 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:45:34.620422 (+ 365us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:45:34.620429 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:45:34.620832 (+ 403us) client_negotiation.cc:769] Sending connection context
0925 02:45:34.621048 (+ 216us) client_negotiation.cc:240] Negotiation successful
0925 02:45:34.621403 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":117,"threads_started":1}
I20260925 02:45:34.624272 27592 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.157.189" port: 46813 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 02:45:34 dist-test-slave-kznw krb5kdc[27436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304334, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.190@KRBTEST.COM for kudu/127.13.157.189@KRBTEST.COM
Sep 25 02:45:34 dist-test-slave-kznw krb5kdc[27436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304334, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.189@KRBTEST.COM for kudu/127.13.157.189@KRBTEST.COM
I20260925 02:45:34.631845 27521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:34.621104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.189:46813 (local address 127.0.0.1:39512)
0925 02:45:34.621210 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:34.621223 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 02:45:34.621292 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:45:34.622509 (+ 1217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:45:34.622512 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:45:34.622678 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:45:34.622776 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:45:34.622781 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:45:34.624069 (+ 1288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:45:34.624073 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:45:34.624907 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:45:34.624919 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:45:34.625782 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:45:34.625785 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:45:34.625876 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:45:34.626341 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:45:34.626354 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:45:34.627845 (+ 1491us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:45:34.629945 (+ 2100us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:45:34.629948 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:45:34.629950 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:45:34.630219 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:45:34.630583 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:45:34.630586 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:45:34.630588 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:45:34.630677 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:45:34.631295 (+ 618us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:45:34.631298 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:45:34.631406 (+ 108us) client_negotiation.cc:769] Sending connection context
0925 02:45:34.631525 (+ 119us) client_negotiation.cc:240] Negotiation successful
0925 02:45:34.631675 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260925 02:45:34.632048 27597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:34.621465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39512 (local address 127.13.157.189:46813)
0925 02:45:34.622123 (+ 658us) server_negotiation.cc:206] Beginning negotiation
0925 02:45:34.622136 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0925 02:45:34.622157 (+ 21us) server_negotiation.cc:407] Connection header received
0925 02:45:34.622198 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:45:34.622203 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:45:34.622388 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:45:34.622509 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:45:34.622879 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:34.623950 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:34.625018 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:34.625664 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:34.625771 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:45:34.627978 (+ 2207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:45:34.627988 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:45:34.628002 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:45:34.628032 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:45:34.629841 (+ 1809us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:45:34.630336 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:45:34.630343 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:45:34.630345 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:45:34.630487 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:45:34.630770 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:45:34.630773 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:45:34.630775 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:45:34.631184 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:45:34.631322 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0925 02:45:34.631680 (+ 358us) server_negotiation.cc:299] Negotiation successful
0925 02:45:34.631862 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":568,"spinlock_wait_cycles":3328,"thread_start_us":142,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.157.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.188.keytab.
Entry for principal kudu/127.13.157.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.188.keytab.
I20260925 02:45:34.633752 27594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:34.624724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.189:46813 (local address 127.0.0.1:39524)
0925 02:45:34.624993 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:34.625007 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 02:45:34.625071 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:45:34.625342 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:45:34.625345 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:45:34.625513 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:45:34.625626 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:45:34.625633 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:45:34.626480 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:45:34.626484 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:45:34.627219 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:45:34.627229 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:45:34.627792 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:45:34.627795 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:45:34.627882 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:45:34.628344 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:45:34.628357 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:45:34.629642 (+ 1285us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:45:34.632414 (+ 2772us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:45:34.632416 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:45:34.632417 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:45:34.632608 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:45:34.632860 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:45:34.632863 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:45:34.632865 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:45:34.632942 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:45:34.633279 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:45:34.633282 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:45:34.633373 (+ 91us) client_negotiation.cc:769] Sending connection context
0925 02:45:34.633463 (+ 90us) client_negotiation.cc:240] Negotiation successful
0925 02:45:34.633584 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197}
I20260925 02:45:34.633854 27598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:34.624763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39524 (local address 127.13.157.189:46813)
0925 02:45:34.625018 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0925 02:45:34.625027 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 02:45:34.625039 (+ 12us) server_negotiation.cc:407] Connection header received
0925 02:45:34.625139 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:45:34.625142 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:45:34.625181 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:45:34.625271 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:45:34.625738 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:34.626366 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:34.627337 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:34.627680 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:34.627770 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:45:34.629780 (+ 2010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:45:34.629783 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:45:34.629785 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:45:34.629819 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:45:34.632317 (+ 2498us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:45:34.632692 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:45:34.632695 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:45:34.632697 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:45:34.632781 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:45:34.633027 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:45:34.633030 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:45:34.633032 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:45:34.633195 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:45:34.633279 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0925 02:45:34.633572 (+ 293us) server_negotiation.cc:299] Negotiation successful
0925 02:45:34.633678 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":194,"spinlock_wait_cycles":5888,"thread_start_us":90,"threads_started":1}
I20260925 02:45:34.634204 27517 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.157.188" port: 41595 } has no permanent_uuid. Determining permanent_uuid...
I20260925 02:45:34.634466 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790304178325258-13942-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790304178325258-13942-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790304178325258-13942-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790304178325258-13942-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.157.188:41595
--webserver_interface=127.13.157.188
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.188.keytab
--principal=kudu/127.13.157.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:35407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.157.190:36279,127.13.157.189:46813,127.13.157.188:41595
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 02:45:34.634729 27592 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.157.188" port: 41595 } has no permanent_uuid. Determining permanent_uuid...
I20260925 02:45:34.634971 27521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:34.634699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.188:41595 (local address 127.0.0.1:44480)
0925 02:45:34.634815 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:34.634870 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.188:41595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260925 02:45:34.635507 27594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:34.635168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.188:41595 (local address 127.0.0.1:44494)
0925 02:45:34.635364 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:34.635442 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.188:41595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131}
W20260925 02:45:34.635689 27541 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.157.188:41595: connect: Connection refused (error 111)
I20260925 02:45:34.635921 27521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:34.635631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.188:41595 (local address 127.0.0.1:44508)
0925 02:45:34.635814 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:34.635861 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.188:41595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125}
W20260925 02:45:34.636035 27517 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.157.188:41595: Network error: Client connection negotiation failed: client connection to 127.13.157.188:41595: connect: Connection refused (error 111)
I20260925 02:45:34.636675 27594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:34.636491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.188:41595 (local address 127.0.0.1:44516)
0925 02:45:34.636587 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:34.636629 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.188:41595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260925 02:45:34.636830 27592 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.157.188:41595: Network error: Client connection negotiation failed: client connection to 127.13.157.188:41595: connect: Connection refused (error 111)
I20260925 02:45:34.667182 27517 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.157.188" port: 41595 } attempt: 1
I20260925 02:45:34.668098 27521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:34.667757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.188:41595 (local address 127.0.0.1:44518)
0925 02:45:34.667945 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:34.668013 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.188:41595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260925 02:45:34.668852 27521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:34.668622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.188:41595 (local address 127.0.0.1:44526)
0925 02:45:34.668733 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:34.668787 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.188:41595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260925 02:45:34.668975 27517 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.157.188:41595: Network error: Client connection negotiation failed: client connection to 127.13.157.188:41595: connect: Connection refused (error 111)
I20260925 02:45:34.686112 27592 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.157.188" port: 41595 } attempt: 1
I20260925 02:45:34.687081 27594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:34.686713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.188:41595 (local address 127.0.0.1:44540)
0925 02:45:34.686934 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:34.686998 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.188:41595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260925 02:45:34.687793 27594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:34.687591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.188:41595 (local address 127.0.0.1:44544)
0925 02:45:34.687688 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:34.687733 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.188:41595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260925 02:45:34.687907 27592 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.157.188:41595: Network error: Client connection negotiation failed: client connection to 127.13.157.188:41595: connect: Connection refused (error 111)
I20260925 02:45:34.744140 27517 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.157.188" port: 41595 } attempt: 2
I20260925 02:45:34.744962 27521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:34.744633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.188:41595 (local address 127.0.0.1:44556)
0925 02:45:34.744822 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:34.744884 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.188:41595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260925 02:45:34.745759 27521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:34.745499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.188:41595 (local address 127.0.0.1:44570)
0925 02:45:34.745615 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:34.745699 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.188:41595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260925 02:45:34.745915 27517 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.157.188:41595: Network error: Client connection negotiation failed: client connection to 127.13.157.188:41595: connect: Connection refused (error 111)
W20260925 02:45:34.747793 27599 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:45:34.748039 27599 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:45:34.748078 27599 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:45:34.752126 27599 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:45:34.752182 27599 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:45:34.752202 27599 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:45:34.752218 27599 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:45:34.752231 27599 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:45:34.756059 27592 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.157.188" port: 41595 } attempt: 2
I20260925 02:45:34.756906 27594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:34.756588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.188:41595 (local address 127.0.0.1:44584)
0925 02:45:34.756761 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:34.756825 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.188:41595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260925 02:45:34.757664 27594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:34.757420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.188:41595 (local address 127.0.0.1:44600)
0925 02:45:34.757534 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:34.757590 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.188:41595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260925 02:45:34.757840 27592 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.157.188:41595: Network error: Client connection negotiation failed: client connection to 127.13.157.188:41595: connect: Connection refused (error 111)
I20260925 02:45:34.758021 27599 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:35407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790304178325258-13942-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790304178325258-13942-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.13.157.190:36279,127.13.157.189:46813,127.13.157.188:41595
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.188.keytab
--principal=kudu/127.13.157.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.157.188:41595
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790304178325258-13942-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.13.157.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27599
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790304178325258-13942-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:45:34.759205 27599 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:45:34.760305 27599 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:45:34.769008 27605 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:45:34.769205 27599 server_base.cc:1034] running on GCE node
W20260925 02:45:34.769011 27608 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:45:34.769008 27606 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:45:34.769887 27599 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:45:34.771116 27599 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:45:34.772310 27599 hybrid_clock.cc:648] HybridClock initialized: now 1790304334772288 us; error 33 us; skew 500 ppm
Sep 25 02:45:34 dist-test-slave-kznw krb5kdc[27436](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304334, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:45:34.775199 27599 init.cc:377] Logged in from keytab as kudu/127.13.157.188@KRBTEST.COM (short username kudu)
I20260925 02:45:34.776182 27599 webserver.cc:460] Webserver started at http://127.13.157.188:40901/ using document root <none> and password file <none>
I20260925 02:45:34.776737 27599 fs_manager.cc:359] Metadata directory not provided
I20260925 02:45:34.776805 27599 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:45:34.777025 27599 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:45:34.779170 27599 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790304178325258-13942-0/minicluster-data/master-2/data/instance:
uuid: "2a7d13744129452a9c0cdc77249ab4ff"
format_stamp: "Formatted at 2026-09-25 02:45:34 on dist-test-slave-kznw"
server_key: "50ed6676ea82357116261e15cd42c3e4"
server_key_iv: "9683b1271ad86a8c13c41ceb65918555"
server_key_version: "encryptionkey@0"
I20260925 02:45:34.779757 27599 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790304178325258-13942-0/minicluster-data/master-2/wal/instance:
uuid: "2a7d13744129452a9c0cdc77249ab4ff"
format_stamp: "Formatted at 2026-09-25 02:45:34 on dist-test-slave-kznw"
server_key: "50ed6676ea82357116261e15cd42c3e4"
server_key_iv: "9683b1271ad86a8c13c41ceb65918555"
server_key_version: "encryptionkey@0"
I20260925 02:45:34.783545 27599 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260925 02:45:34.786159 27615 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:45:34.787508 27599 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 02:45:34.787662 27599 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790304178325258-13942-0/minicluster-data/master-2/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790304178325258-13942-0/minicluster-data/master-2/wal
uuid: "2a7d13744129452a9c0cdc77249ab4ff"
format_stamp: "Formatted at 2026-09-25 02:45:34 on dist-test-slave-kznw"
server_key: "50ed6676ea82357116261e15cd42c3e4"
server_key_iv: "9683b1271ad86a8c13c41ceb65918555"
server_key_version: "encryptionkey@0"
I20260925 02:45:34.787770 27599 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790304178325258-13942-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790304178325258-13942-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790304178325258-13942-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:45:34.817530 27599 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:45:34.821014 27599 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:45:34.821206 27599 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:45:34.828403 27599 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.188:41595
I20260925 02:45:34.828670 27667 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.188:41595 every 8 connection(s)
I20260925 02:45:34.829449 27599 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790304178325258-13942-0/minicluster-data/master-2/data/info.pb
I20260925 02:45:34.831272 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 27599
I20260925 02:45:34.831388 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790304178325258-13942-0/minicluster-data/master-2/wal/instance
I20260925 02:45:34.831701 13942 external_mini_cluster.cc:1428] Setting key 7ac74c5cc0a81f5b3c0c343fe768e9ce
I20260925 02:45:34.832600 27668 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:45:34.837234 27668 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.157.190" port: 36279 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 02:45:34 dist-test-slave-kznw krb5kdc[27436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304334, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.157.188@KRBTEST.COM
Sep 25 02:45:34 dist-test-slave-kznw krb5kdc[27436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304334, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.188@KRBTEST.COM for kudu/127.13.157.190@KRBTEST.COM
I20260925 02:45:34.845072 27517 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.157.188" port: 41595 } attempt: 3
I20260925 02:45:34.846472 27671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:34.833018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44614 (local address 127.13.157.188:41595)
0925 02:45:34.833447 (+ 429us) server_negotiation.cc:206] Beginning negotiation
0925 02:45:34.833466 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0925 02:45:34.833509 (+ 43us) server_negotiation.cc:407] Connection header received
0925 02:45:34.834337 (+ 828us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:45:34.834360 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:45:34.834790 (+ 430us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:45:34.835189 (+ 399us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:45:34.835911 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:34.837340 (+ 1429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:34.838282 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:34.838887 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:34.839482 (+ 595us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:45:34.841548 (+ 2066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:45:34.841553 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:45:34.841562 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:45:34.841589 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:45:34.843381 (+ 1792us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:45:34.843783 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:45:34.843788 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:45:34.843793 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:45:34.843900 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:45:34.844177 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:45:34.844180 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:45:34.844181 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:45:34.844533 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:45:34.844690 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0925 02:45:34.845078 (+ 388us) server_negotiation.cc:299] Negotiation successful
0925 02:45:34.845543 (+ 465us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":110,"threads_started":1}
I20260925 02:45:34.848614 27520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:34.839109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39614 (local address 127.13.157.190:36279)
0925 02:45:34.839249 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0925 02:45:34.839254 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:45:34.839550 (+ 296us) server_negotiation.cc:407] Connection header received
0925 02:45:34.839823 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:45:34.839827 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:45:34.839879 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:45:34.839965 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:45:34.840567 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:34.841074 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:34.841988 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:34.842321 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:34.842422 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:45:34.844544 (+ 2122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:45:34.844548 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:45:34.844551 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:45:34.844578 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:45:34.846542 (+ 1964us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:45:34.847385 (+ 843us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:45:34.847388 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:45:34.847389 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:45:34.847475 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:45:34.847755 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:45:34.847758 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:45:34.847760 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:45:34.847927 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:45:34.848009 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0925 02:45:34.848357 (+ 348us) server_negotiation.cc:299] Negotiation successful
0925 02:45:34.848475 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260925 02:45:34.848610 27672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:34.839059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:36279 (local address 127.0.0.1:39614)
0925 02:45:34.839418 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:34.839473 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0925 02:45:34.839704 (+ 231us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:45:34.840002 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:45:34.840006 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:45:34.840322 (+ 316us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:45:34.840446 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:45:34.840451 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:45:34.841209 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:45:34.841211 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:45:34.841860 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:45:34.841871 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:45:34.842431 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:45:34.842435 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:45:34.842553 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:45:34.843053 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:45:34.843065 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:45:34.844418 (+ 1353us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:45:34.846994 (+ 2576us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:45:34.847002 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:45:34.847008 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:45:34.847270 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:45:34.847574 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:45:34.847576 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:45:34.847577 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:45:34.847657 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:45:34.848024 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:45:34.848030 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:45:34.848132 (+ 102us) client_negotiation.cc:769] Sending connection context
0925 02:45:34.848259 (+ 127us) client_negotiation.cc:240] Negotiation successful
0925 02:45:34.848420 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":116,"threads_started":1}
I20260925 02:45:34.848980 27592 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.157.188" port: 41595 } attempt: 3
I20260925 02:45:34.850178 27668 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.157.189" port: 46813 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 02:45:34 dist-test-slave-kznw krb5kdc[27436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304334, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.190@KRBTEST.COM for kudu/127.13.157.188@KRBTEST.COM
Sep 25 02:45:34 dist-test-slave-kznw krb5kdc[27436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304334, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.189@KRBTEST.COM for kudu/127.13.157.188@KRBTEST.COM
I20260925 02:45:34.855139 27521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:34.845465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.188:41595 (local address 127.0.0.1:44630)
0925 02:45:34.845582 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:34.845596 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 02:45:34.845680 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:45:34.846190 (+ 510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:45:34.846193 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:45:34.846453 (+ 260us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:45:34.846557 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:45:34.846565 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:45:34.847369 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:45:34.847372 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:45:34.848081 (+ 709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:45:34.848092 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:45:34.848645 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:45:34.848649 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:45:34.848738 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:45:34.849268 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:45:34.849282 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:45:34.850712 (+ 1430us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:45:34.853578 (+ 2866us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:45:34.853581 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:45:34.853583 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:45:34.853811 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:45:34.854139 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:45:34.854142 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:45:34.854144 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:45:34.854233 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:45:34.854642 (+ 409us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:45:34.854645 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:45:34.854740 (+ 95us) client_negotiation.cc:769] Sending connection context
0925 02:45:34.854822 (+ 82us) client_negotiation.cc:240] Negotiation successful
0925 02:45:34.854963 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
Sep 25 02:45:34 dist-test-slave-kznw krb5kdc[27436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304334, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.188@KRBTEST.COM for kudu/127.13.157.189@KRBTEST.COM
I20260925 02:45:34.855773 27673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:34.845635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44630 (local address 127.13.157.188:41595)
0925 02:45:34.845959 (+ 324us) server_negotiation.cc:206] Beginning negotiation
0925 02:45:34.845968 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 02:45:34.845981 (+ 13us) server_negotiation.cc:407] Connection header received
0925 02:45:34.846013 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:45:34.846016 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:45:34.846054 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:45:34.846270 (+ 216us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:45:34.846704 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:34.847256 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:34.848203 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:34.848513 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:34.848934 (+ 421us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:45:34.850824 (+ 1890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:45:34.850828 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:45:34.850830 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:45:34.850853 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:45:34.853459 (+ 2606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:45:34.853929 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:45:34.853932 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:45:34.853933 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:45:34.854026 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:45:34.854350 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:45:34.854353 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:45:34.854354 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:45:34.854531 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:45:34.855419 (+ 888us) server_negotiation.cc:1035] Waiting for connection context
0925 02:45:34.855501 (+ 82us) server_negotiation.cc:299] Negotiation successful
0925 02:45:34.855609 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":259,"thread_start_us":142,"threads_started":1}
Sep 25 02:45:34 dist-test-slave-kznw krb5kdc[27436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304334, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.157.190@KRBTEST.COM
I20260925 02:45:34.858635 27594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:34.849393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.188:41595 (local address 127.0.0.1:44646)
0925 02:45:34.849670 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:34.849685 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 02:45:34.849769 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:45:34.850376 (+ 607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:45:34.850379 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:45:34.850555 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:45:34.850667 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:45:34.850675 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:45:34.851727 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:45:34.851730 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:45:34.852743 (+ 1013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:45:34.852752 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:45:34.853324 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:45:34.853328 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:45:34.853410 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:45:34.853831 (+ 421us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:45:34.853844 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:45:34.855247 (+ 1403us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:45:34.856991 (+ 1744us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:45:34.856994 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:45:34.856996 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:45:34.857399 (+ 403us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:45:34.857671 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:45:34.857674 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:45:34.857676 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:45:34.857754 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:45:34.858163 (+ 409us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:45:34.858166 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:45:34.858257 (+ 91us) client_negotiation.cc:769] Sending connection context
0925 02:45:34.858342 (+ 85us) client_negotiation.cc:240] Negotiation successful
0925 02:45:34.858477 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"mutex_wait_us":348}
I20260925 02:45:34.858700 27671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:34.849643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44646 (local address 127.13.157.188:41595)
0925 02:45:34.850146 (+ 503us) server_negotiation.cc:206] Beginning negotiation
0925 02:45:34.850151 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:45:34.850165 (+ 14us) server_negotiation.cc:407] Connection header received
0925 02:45:34.850196 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:45:34.850199 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:45:34.850238 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:45:34.850783 (+ 545us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:45:34.850864 (+ 81us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:34.851630 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:34.852861 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:34.853212 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:34.853626 (+ 414us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:45:34.855384 (+ 1758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:45:34.855387 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:45:34.855389 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:45:34.855416 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:45:34.856872 (+ 1456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:45:34.857477 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:45:34.857480 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:45:34.857482 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:45:34.857563 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:45:34.857872 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:45:34.857875 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:45:34.857877 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:45:34.858037 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:45:34.858162 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0925 02:45:34.858409 (+ 247us) server_negotiation.cc:299] Negotiation successful
0925 02:45:34.858568 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":43}
I20260925 02:45:34.859222 27517 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f34aeb99aad45a2a58463a851771b79: Bootstrap starting.
I20260925 02:45:34.861622 27672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:34.850582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.189:46813 (local address 127.0.0.1:39530)
0925 02:45:34.850831 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:34.850843 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 02:45:34.850969 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:45:34.851297 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:45:34.851299 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:45:34.851469 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:45:34.851574 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:45:34.851580 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:45:34.852656 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:45:34.852660 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:45:34.853439 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:45:34.853450 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:45:34.854126 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:45:34.854129 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:45:34.854217 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:45:34.854587 (+ 370us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:45:34.854600 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:45:34.856413 (+ 1813us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:45:34.860107 (+ 3694us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:45:34.860110 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:45:34.860112 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:45:34.860340 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:45:34.860622 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:45:34.860625 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:45:34.860633 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:45:34.860725 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:45:34.861169 (+ 444us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:45:34.861172 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:45:34.861260 (+ 88us) client_negotiation.cc:769] Sending connection context
0925 02:45:34.861368 (+ 108us) client_negotiation.cc:240] Negotiation successful
0925 02:45:34.861487 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191}
I20260925 02:45:34.861681 27598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:34.850657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39530 (local address 127.13.157.189:46813)
0925 02:45:34.850755 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0925 02:45:34.850759 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:45:34.850892 (+ 133us) server_negotiation.cc:407] Connection header received
0925 02:45:34.851072 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:45:34.851075 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:45:34.851118 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:45:34.851168 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:45:34.851721 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:34.852541 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:34.853577 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:34.853923 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:34.853991 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:45:34.856548 (+ 2557us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:45:34.856551 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:45:34.856553 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:45:34.856578 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:45:34.859994 (+ 3416us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:45:34.860436 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:45:34.860439 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:45:34.860441 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:45:34.860530 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:45:34.860840 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:45:34.860843 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:45:34.860845 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:45:34.861067 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:45:34.861173 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0925 02:45:34.861429 (+ 256us) server_negotiation.cc:299] Negotiation successful
0925 02:45:34.861542 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":54,"server-negotiator.queue_time_us":41}
I20260925 02:45:34.861755 27592 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 894f7d93b7df4cd19c802e234a6c761b: Bootstrap starting.
I20260925 02:45:34.862147 27517 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7f34aeb99aad45a2a58463a851771b79: Neither blocks nor log segments found. Creating new log.
I20260925 02:45:34.862892 27517 log.cc:824] T 00000000000000000000000000000000 P 7f34aeb99aad45a2a58463a851771b79: Log is configured to *not* fsync() on all Append() calls
I20260925 02:45:34.863330 27668 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.157.188" port: 41595 } has no permanent_uuid. Determining permanent_uuid...
I20260925 02:45:34.864332 27592 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 894f7d93b7df4cd19c802e234a6c761b: Neither blocks nor log segments found. Creating new log.
I20260925 02:45:34.865260 27592 log.cc:824] T 00000000000000000000000000000000 P 894f7d93b7df4cd19c802e234a6c761b: Log is configured to *not* fsync() on all Append() calls
I20260925 02:45:34.865656 27517 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f34aeb99aad45a2a58463a851771b79: No bootstrap required, opened a new log
I20260925 02:45:34.866943 27520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:34.849336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39620 (local address 127.13.157.190:36279)
0925 02:45:34.850802 (+ 1466us) server_negotiation.cc:206] Beginning negotiation
0925 02:45:34.850806 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:45:34.850815 (+ 9us) server_negotiation.cc:407] Connection header received
0925 02:45:34.850839 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:45:34.850842 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:45:34.850879 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:45:34.852119 (+ 1240us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:45:34.852201 (+ 82us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:34.853145 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:34.854706 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:34.855049 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:34.857278 (+ 2229us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:45:34.858930 (+ 1652us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:45:34.858935 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:45:34.858937 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:45:34.858962 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:45:34.864288 (+ 5326us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:45:34.864899 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:45:34.864902 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:45:34.864903 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:45:34.864988 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:45:34.865319 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:45:34.865322 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:45:34.865324 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:45:34.865584 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:45:34.866567 (+ 983us) server_negotiation.cc:1035] Waiting for connection context
0925 02:45:34.866646 (+ 79us) server_negotiation.cc:299] Negotiation successful
0925 02:45:34.866762 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":415}
I20260925 02:45:34.867338 27592 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 894f7d93b7df4cd19c802e234a6c761b: No bootstrap required, opened a new log
Sep 25 02:45:34 dist-test-slave-kznw krb5kdc[27436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304334, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.188@KRBTEST.COM for kudu/127.13.157.188@KRBTEST.COM
I20260925 02:45:34.869786 27517 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7f34aeb99aad45a2a58463a851771b79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f34aeb99aad45a2a58463a851771b79" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 36279 } } peers { permanent_uuid: "894f7d93b7df4cd19c802e234a6c761b" member_type: VOTER last_known_addr { host: "127.13.157.189" port: 46813 } } peers { permanent_uuid: "2a7d13744129452a9c0cdc77249ab4ff" member_type: VOTER last_known_addr { host: "127.13.157.188" port: 41595 } }
I20260925 02:45:34.870033 27517 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7f34aeb99aad45a2a58463a851771b79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:45:34.870100 27517 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7f34aeb99aad45a2a58463a851771b79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f34aeb99aad45a2a58463a851771b79, State: Initialized, Role: FOLLOWER
I20260925 02:45:34.870718 27517 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7f34aeb99aad45a2a58463a851771b79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f34aeb99aad45a2a58463a851771b79" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 36279 } } peers { permanent_uuid: "894f7d93b7df4cd19c802e234a6c761b" member_type: VOTER last_known_addr { host: "127.13.157.189" port: 46813 } } peers { permanent_uuid: "2a7d13744129452a9c0cdc77249ab4ff" member_type: VOTER last_known_addr { host: "127.13.157.188" port: 41595 } }
I20260925 02:45:34.871229 27592 raft_consensus.cc:348] T 00000000000000000000000000000000 P 894f7d93b7df4cd19c802e234a6c761b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f34aeb99aad45a2a58463a851771b79" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 36279 } } peers { permanent_uuid: "894f7d93b7df4cd19c802e234a6c761b" member_type: VOTER last_known_addr { host: "127.13.157.189" port: 46813 } } peers { permanent_uuid: "2a7d13744129452a9c0cdc77249ab4ff" member_type: VOTER last_known_addr { host: "127.13.157.188" port: 41595 } }
I20260925 02:45:34.871417 27592 raft_consensus.cc:374] T 00000000000000000000000000000000 P 894f7d93b7df4cd19c802e234a6c761b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:45:34.871455 27592 raft_consensus.cc:729] T 00000000000000000000000000000000 P 894f7d93b7df4cd19c802e234a6c761b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 894f7d93b7df4cd19c802e234a6c761b, State: Initialized, Role: FOLLOWER
I20260925 02:45:34.871924 27592 consensus_queue.cc:260] T 00000000000000000000000000000000 P 894f7d93b7df4cd19c802e234a6c761b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f34aeb99aad45a2a58463a851771b79" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 36279 } } peers { permanent_uuid: "894f7d93b7df4cd19c802e234a6c761b" member_type: VOTER last_known_addr { host: "127.13.157.189" port: 46813 } } peers { permanent_uuid: "2a7d13744129452a9c0cdc77249ab4ff" member_type: VOTER last_known_addr { host: "127.13.157.188" port: 41595 } }
I20260925 02:45:34.872063 27672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:34.863772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.188:41595 (local address 127.0.0.1:44662)
0925 02:45:34.864046 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:34.864059 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 02:45:34.864146 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:45:34.864419 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:45:34.864422 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:45:34.864568 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:45:34.864672 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:45:34.864679 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:45:34.865591 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:45:34.865594 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:45:34.866331 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:45:34.866342 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:45:34.866856 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:45:34.866860 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:45:34.866971 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:45:34.867401 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:45:34.867415 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:45:34.868831 (+ 1416us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:45:34.870693 (+ 1862us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:45:34.870696 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:45:34.870698 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:45:34.870941 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:45:34.871212 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:45:34.871215 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:45:34.871217 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:45:34.871288 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:45:34.871643 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:45:34.871646 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:45:34.871721 (+ 75us) client_negotiation.cc:769] Sending connection context
0925 02:45:34.871799 (+ 78us) client_negotiation.cc:240] Negotiation successful
0925 02:45:34.871918 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214}
I20260925 02:45:34.872346 27517 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7f34aeb99aad45a2a58463a851771b79 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:45:34.872526 27671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:34.863953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44662 (local address 127.13.157.188:41595)
0925 02:45:34.864048 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0925 02:45:34.864052 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:45:34.864098 (+ 46us) server_negotiation.cc:407] Connection header received
0925 02:45:34.864279 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:45:34.864282 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:45:34.864321 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:45:34.864761 (+ 440us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:45:34.864822 (+ 61us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:34.865487 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:34.866457 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:34.866749 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:34.868951 (+ 2202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:45:34.869043 (+ 92us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:45:34.869046 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:45:34.869048 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:45:34.869064 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:45:34.870568 (+ 1504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:45:34.871046 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:45:34.871048 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:45:34.871050 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:45:34.871128 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:45:34.871370 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:45:34.871372 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:45:34.871373 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:45:34.871543 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:45:34.872233 (+ 690us) server_negotiation.cc:1035] Waiting for connection context
0925 02:45:34.872304 (+ 71us) server_negotiation.cc:299] Negotiation successful
0925 02:45:34.872400 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260925 02:45:34.873193 27592 sys_catalog.cc:564] T 00000000000000000000000000000000 P 894f7d93b7df4cd19c802e234a6c761b [sys.catalog]: configured and running, proceeding with master startup.
Sep 25 02:45:34 dist-test-slave-kznw krb5kdc[27436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304334, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.157.189@KRBTEST.COM
I20260925 02:45:34.875592 27668 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a7d13744129452a9c0cdc77249ab4ff: Bootstrap starting.
I20260925 02:45:34.875617 27677 sys_catalog.cc:455] T 00000000000000000000000000000000 P 894f7d93b7df4cd19c802e234a6c761b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f34aeb99aad45a2a58463a851771b79" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 36279 } } peers { permanent_uuid: "894f7d93b7df4cd19c802e234a6c761b" member_type: VOTER last_known_addr { host: "127.13.157.189" port: 46813 } } peers { permanent_uuid: "2a7d13744129452a9c0cdc77249ab4ff" member_type: VOTER last_known_addr { host: "127.13.157.188" port: 41595 } } }
I20260925 02:45:34.875763 27677 sys_catalog.cc:458] T 00000000000000000000000000000000 P 894f7d93b7df4cd19c802e234a6c761b [sys.catalog]: This master's current role is: FOLLOWER
I20260925 02:45:34.876369 27676 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f34aeb99aad45a2a58463a851771b79 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f34aeb99aad45a2a58463a851771b79" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 36279 } } peers { permanent_uuid: "894f7d93b7df4cd19c802e234a6c761b" member_type: VOTER last_known_addr { host: "127.13.157.189" port: 46813 } } peers { permanent_uuid: "2a7d13744129452a9c0cdc77249ab4ff" member_type: VOTER last_known_addr { host: "127.13.157.188" port: 41595 } } }
I20260925 02:45:34.876557 27676 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f34aeb99aad45a2a58463a851771b79 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 02:45:34.878350 27668 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2a7d13744129452a9c0cdc77249ab4ff: Neither blocks nor log segments found. Creating new log.
W20260925 02:45:34.879093 27470 tablet.cc:2369] T 00000000000000000000000000000000 P 7f34aeb99aad45a2a58463a851771b79: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 02:45:34.878964 27692 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 894f7d93b7df4cd19c802e234a6c761b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260925 02:45:34.879443 27598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:34.868388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39544 (local address 127.13.157.189:46813)
0925 02:45:34.868781 (+ 393us) server_negotiation.cc:206] Beginning negotiation
0925 02:45:34.868785 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:45:34.868798 (+ 13us) server_negotiation.cc:407] Connection header received
0925 02:45:34.868830 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:45:34.868833 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:45:34.868872 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:45:34.868935 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:45:34.869717 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:34.870654 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:34.871814 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:34.872362 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:34.872539 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:45:34.875957 (+ 3418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:45:34.875960 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:45:34.875963 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:45:34.875979 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:45:34.877469 (+ 1490us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:45:34.878035 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:45:34.878038 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:45:34.878039 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:45:34.878126 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:45:34.878403 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:45:34.878406 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:45:34.878408 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:45:34.878611 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:45:34.878815 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0925 02:45:34.879166 (+ 351us) server_negotiation.cc:299] Negotiation successful
0925 02:45:34.879279 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323}
I20260925 02:45:34.880031 27668 log.cc:824] T 00000000000000000000000000000000 P 2a7d13744129452a9c0cdc77249ab4ff: Log is configured to *not* fsync() on all Append() calls
W20260925 02:45:34.880215 27692 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 02:45:34.881104 27524 master_runner.cc:186] Error getting master registration for 127.13.157.190:36279: OK, instance_id { permanent_uuid: "7f34aeb99aad45a2a58463a851771b79" instance_seqno: 1790304334379282 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260925 02:45:34.884559 27668 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a7d13744129452a9c0cdc77249ab4ff: No bootstrap required, opened a new log
I20260925 02:45:34.884675 27524 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260925 02:45:34.884994 13942 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 02:45:34.888484 27668 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2a7d13744129452a9c0cdc77249ab4ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f34aeb99aad45a2a58463a851771b79" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 36279 } } peers { permanent_uuid: "894f7d93b7df4cd19c802e234a6c761b" member_type: VOTER last_known_addr { host: "127.13.157.189" port: 46813 } } peers { permanent_uuid: "2a7d13744129452a9c0cdc77249ab4ff" member_type: VOTER last_known_addr { host: "127.13.157.188" port: 41595 } }
I20260925 02:45:34.888741 27668 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2a7d13744129452a9c0cdc77249ab4ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:45:34.888797 27668 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2a7d13744129452a9c0cdc77249ab4ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a7d13744129452a9c0cdc77249ab4ff, State: Initialized, Role: FOLLOWER
I20260925 02:45:34.889315 27668 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2a7d13744129452a9c0cdc77249ab4ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f34aeb99aad45a2a58463a851771b79" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 36279 } } peers { permanent_uuid: "894f7d93b7df4cd19c802e234a6c761b" member_type: VOTER last_known_addr { host: "127.13.157.189" port: 46813 } } peers { permanent_uuid: "2a7d13744129452a9c0cdc77249ab4ff" member_type: VOTER last_known_addr { host: "127.13.157.188" port: 41595 } }
I20260925 02:45:34.890673 27668 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2a7d13744129452a9c0cdc77249ab4ff [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:45:34.887825 27449 master_runner.cc:186] Error getting master registration for 127.13.157.188:41595: OK, instance_id { permanent_uuid: "2a7d13744129452a9c0cdc77249ab4ff" instance_seqno: 1790304334826373 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260925 02:45:34.892117 27449 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260925 02:45:34.894843 27701 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7f34aeb99aad45a2a58463a851771b79: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 02:45:34.894938 27701 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 02:45:34.895380 27709 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a7d13744129452a9c0cdc77249ab4ff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f34aeb99aad45a2a58463a851771b79" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 36279 } } peers { permanent_uuid: "894f7d93b7df4cd19c802e234a6c761b" member_type: VOTER last_known_addr { host: "127.13.157.189" port: 46813 } } peers { permanent_uuid: "2a7d13744129452a9c0cdc77249ab4ff" member_type: VOTER last_known_addr { host: "127.13.157.188" port: 41595 } } }
I20260925 02:45:34.895514 27709 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a7d13744129452a9c0cdc77249ab4ff [sys.catalog]: This master's current role is: FOLLOWER
W20260925 02:45:34.896484 27720 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2a7d13744129452a9c0cdc77249ab4ff: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260925 02:45:34.896541 27520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:34.886956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39622 (local address 127.13.157.190:36279)
0925 02:45:34.887119 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0925 02:45:34.887123 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:45:34.887556 (+ 433us) server_negotiation.cc:407] Connection header received
0925 02:45:34.887748 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:45:34.887752 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:45:34.887799 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:45:34.887847 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:45:34.888805 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:34.889584 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:34.890761 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:34.891115 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:34.891193 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:45:34.893144 (+ 1951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:45:34.893148 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:45:34.893151 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:45:34.893178 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:45:34.894739 (+ 1561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:45:34.895277 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:45:34.895282 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:45:34.895284 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:45:34.895378 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:45:34.895683 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:45:34.895687 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:45:34.895689 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:45:34.895876 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:45:34.895992 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0925 02:45:34.896270 (+ 278us) server_negotiation.cc:299] Negotiation successful
0925 02:45:34.896391 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
W20260925 02:45:34.896770 27720 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 02:45:34.899049 27599 master_runner.cc:263] No leader master found from master 2a7d13744129452a9c0cdc77249ab4ff
I20260925 02:45:34.899120 27599 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260925 02:45:34.906083 27598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:34.886994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39556 (local address 127.13.157.189:46813)
0925 02:45:34.887100 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0925 02:45:34.887104 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:45:34.897140 (+ 10036us) server_negotiation.cc:407] Connection header received
0925 02:45:34.897353 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:45:34.897357 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:45:34.897409 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:45:34.897459 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:45:34.898500 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:34.899242 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:34.900417 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:34.900739 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:34.900814 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:45:34.902456 (+ 1642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:45:34.902460 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:45:34.902463 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:45:34.902491 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:45:34.904178 (+ 1687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:45:34.904693 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:45:34.904697 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:45:34.904699 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:45:34.904786 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:45:34.905093 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:45:34.905097 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:45:34.905098 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:45:34.905315 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:45:34.905418 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0925 02:45:34.905795 (+ 377us) server_negotiation.cc:299] Negotiation successful
0925 02:45:34.905916 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260925 02:45:34.911737 27677 raft_consensus.cc:482] T 00000000000000000000000000000000 P 894f7d93b7df4cd19c802e234a6c761b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 02:45:34.912221 27677 raft_consensus.cc:504] T 00000000000000000000000000000000 P 894f7d93b7df4cd19c802e234a6c761b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f34aeb99aad45a2a58463a851771b79" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 36279 } } peers { permanent_uuid: "894f7d93b7df4cd19c802e234a6c761b" member_type: VOTER last_known_addr { host: "127.13.157.189" port: 46813 } } peers { permanent_uuid: "2a7d13744129452a9c0cdc77249ab4ff" member_type: VOTER last_known_addr { host: "127.13.157.188" port: 41595 } }
I20260925 02:45:34.913558 27677 leader_election.cc:288] T 00000000000000000000000000000000 P 894f7d93b7df4cd19c802e234a6c761b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7f34aeb99aad45a2a58463a851771b79 (127.13.157.190:36279), 2a7d13744129452a9c0cdc77249ab4ff (127.13.157.188:41595)
I20260925 02:45:34.914351 27642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "894f7d93b7df4cd19c802e234a6c761b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a7d13744129452a9c0cdc77249ab4ff" is_pre_election: true
I20260925 02:45:34.914355 27491 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "894f7d93b7df4cd19c802e234a6c761b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f34aeb99aad45a2a58463a851771b79" is_pre_election: true
I20260925 02:45:34.914693 27642 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2a7d13744129452a9c0cdc77249ab4ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 894f7d93b7df4cd19c802e234a6c761b in term 0.
I20260925 02:45:34.914693 27491 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7f34aeb99aad45a2a58463a851771b79 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 894f7d93b7df4cd19c802e234a6c761b in term 0.
I20260925 02:45:34.915191 27540 leader_election.cc:302] T 00000000000000000000000000000000 P 894f7d93b7df4cd19c802e234a6c761b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2a7d13744129452a9c0cdc77249ab4ff, 7f34aeb99aad45a2a58463a851771b79, 894f7d93b7df4cd19c802e234a6c761b; no voters:
I20260925 02:45:34.915706 27677 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 894f7d93b7df4cd19c802e234a6c761b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 02:45:34.915795 27677 raft_consensus.cc:482] T 00000000000000000000000000000000 P 894f7d93b7df4cd19c802e234a6c761b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 02:45:34.915843 27677 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 894f7d93b7df4cd19c802e234a6c761b [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:45:34.916747 27671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:34.896412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44666 (local address 127.13.157.188:41595)
0925 02:45:34.896639 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0925 02:45:34.896643 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:45:34.903089 (+ 6446us) server_negotiation.cc:407] Connection header received
0925 02:45:34.903206 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:45:34.903209 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:45:34.903256 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:45:34.903323 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:45:34.904096 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:34.904832 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:34.905928 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:34.906273 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:34.906352 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:45:34.911501 (+ 5149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:45:34.911507 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:45:34.911509 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:45:34.911542 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:45:34.913294 (+ 1752us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:45:34.913828 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:45:34.913833 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:45:34.913834 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:45:34.915294 (+ 1460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:45:34.915662 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:45:34.915666 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:45:34.915668 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:45:34.915913 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:45:34.916031 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0925 02:45:34.916442 (+ 411us) server_negotiation.cc:299] Negotiation successful
0925 02:45:34.916572 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148}
I20260925 02:45:34.917200 27677 raft_consensus.cc:504] T 00000000000000000000000000000000 P 894f7d93b7df4cd19c802e234a6c761b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f34aeb99aad45a2a58463a851771b79" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 36279 } } peers { permanent_uuid: "894f7d93b7df4cd19c802e234a6c761b" member_type: VOTER last_known_addr { host: "127.13.157.189" port: 46813 } } peers { permanent_uuid: "2a7d13744129452a9c0cdc77249ab4ff" member_type: VOTER last_known_addr { host: "127.13.157.188" port: 41595 } }
I20260925 02:45:34.917577 27677 leader_election.cc:288] T 00000000000000000000000000000000 P 894f7d93b7df4cd19c802e234a6c761b [CANDIDATE]: Term 1 election: Requested vote from peers 7f34aeb99aad45a2a58463a851771b79 (127.13.157.190:36279), 2a7d13744129452a9c0cdc77249ab4ff (127.13.157.188:41595)
I20260925 02:45:34.918062 27491 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "894f7d93b7df4cd19c802e234a6c761b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f34aeb99aad45a2a58463a851771b79"
I20260925 02:45:34.918210 27491 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7f34aeb99aad45a2a58463a851771b79 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:45:34.918231 27642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "894f7d93b7df4cd19c802e234a6c761b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a7d13744129452a9c0cdc77249ab4ff"
I20260925 02:45:34.918340 27642 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2a7d13744129452a9c0cdc77249ab4ff [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:45:34.919549 27491 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7f34aeb99aad45a2a58463a851771b79 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 894f7d93b7df4cd19c802e234a6c761b in term 1.
I20260925 02:45:34.919661 27642 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2a7d13744129452a9c0cdc77249ab4ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 894f7d93b7df4cd19c802e234a6c761b in term 1.
I20260925 02:45:34.919878 27540 leader_election.cc:302] T 00000000000000000000000000000000 P 894f7d93b7df4cd19c802e234a6c761b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f34aeb99aad45a2a58463a851771b79, 894f7d93b7df4cd19c802e234a6c761b; no voters:
I20260925 02:45:34.920101 27677 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 894f7d93b7df4cd19c802e234a6c761b [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:45:34.920324 27677 raft_consensus.cc:686] T 00000000000000000000000000000000 P 894f7d93b7df4cd19c802e234a6c761b [term 1 LEADER]: Becoming Leader. State: Replica: 894f7d93b7df4cd19c802e234a6c761b, State: Running, Role: LEADER
I20260925 02:45:34.920687 27677 consensus_queue.cc:237] T 00000000000000000000000000000000 P 894f7d93b7df4cd19c802e234a6c761b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f34aeb99aad45a2a58463a851771b79" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 36279 } } peers { permanent_uuid: "894f7d93b7df4cd19c802e234a6c761b" member_type: VOTER last_known_addr { host: "127.13.157.189" port: 46813 } } peers { permanent_uuid: "2a7d13744129452a9c0cdc77249ab4ff" member_type: VOTER last_known_addr { host: "127.13.157.188" port: 41595 } }
I20260925 02:45:34.924119 27722 sys_catalog.cc:455] T 00000000000000000000000000000000 P 894f7d93b7df4cd19c802e234a6c761b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 894f7d93b7df4cd19c802e234a6c761b. Latest consensus state: current_term: 1 leader_uuid: "894f7d93b7df4cd19c802e234a6c761b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f34aeb99aad45a2a58463a851771b79" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 36279 } } peers { permanent_uuid: "894f7d93b7df4cd19c802e234a6c761b" member_type: VOTER last_known_addr { host: "127.13.157.189" port: 46813 } } peers { permanent_uuid: "2a7d13744129452a9c0cdc77249ab4ff" member_type: VOTER last_known_addr { host: "127.13.157.188" port: 41595 } } }
I20260925 02:45:34.924320 27722 sys_catalog.cc:458] T 00000000000000000000000000000000 P 894f7d93b7df4cd19c802e234a6c761b [sys.catalog]: This master's current role is: LEADER
I20260925 02:45:34.926123 27724 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:45:34.927203 27724 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:45:34.936111 27642 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2a7d13744129452a9c0cdc77249ab4ff [term 1 FOLLOWER]: Refusing update from remote peer 894f7d93b7df4cd19c802e234a6c761b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:45:34.939208 27722 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 894f7d93b7df4cd19c802e234a6c761b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a7d13744129452a9c0cdc77249ab4ff" member_type: VOTER last_known_addr { host: "127.13.157.188" port: 41595 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:45:34.943539 27709 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a7d13744129452a9c0cdc77249ab4ff [sys.catalog]: SysCatalogTable state changed. Reason: New leader 894f7d93b7df4cd19c802e234a6c761b. Latest consensus state: current_term: 1 leader_uuid: "894f7d93b7df4cd19c802e234a6c761b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f34aeb99aad45a2a58463a851771b79" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 36279 } } peers { permanent_uuid: "894f7d93b7df4cd19c802e234a6c761b" member_type: VOTER last_known_addr { host: "127.13.157.189" port: 46813 } } peers { permanent_uuid: "2a7d13744129452a9c0cdc77249ab4ff" member_type: VOTER last_known_addr { host: "127.13.157.188" port: 41595 } } }
I20260925 02:45:34.943663 27709 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a7d13744129452a9c0cdc77249ab4ff [sys.catalog]: This master's current role is: FOLLOWER
I20260925 02:45:34.944765 27491 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7f34aeb99aad45a2a58463a851771b79 [term 1 FOLLOWER]: Refusing update from remote peer 894f7d93b7df4cd19c802e234a6c761b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:45:34.945464 27722 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 894f7d93b7df4cd19c802e234a6c761b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f34aeb99aad45a2a58463a851771b79" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 36279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:45:34.949179 27676 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f34aeb99aad45a2a58463a851771b79 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 894f7d93b7df4cd19c802e234a6c761b. Latest consensus state: current_term: 1 leader_uuid: "894f7d93b7df4cd19c802e234a6c761b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f34aeb99aad45a2a58463a851771b79" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 36279 } } peers { permanent_uuid: "894f7d93b7df4cd19c802e234a6c761b" member_type: VOTER last_known_addr { host: "127.13.157.189" port: 46813 } } peers { permanent_uuid: "2a7d13744129452a9c0cdc77249ab4ff" member_type: VOTER last_known_addr { host: "127.13.157.188" port: 41595 } } }
I20260925 02:45:34.949294 27676 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f34aeb99aad45a2a58463a851771b79 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 02:45:34.956136 27677 sys_catalog.cc:455] T 00000000000000000000000000000000 P 894f7d93b7df4cd19c802e234a6c761b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "894f7d93b7df4cd19c802e234a6c761b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f34aeb99aad45a2a58463a851771b79" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 36279 } } peers { permanent_uuid: "894f7d93b7df4cd19c802e234a6c761b" member_type: VOTER last_known_addr { host: "127.13.157.189" port: 46813 } } peers { permanent_uuid: "2a7d13744129452a9c0cdc77249ab4ff" member_type: VOTER last_known_addr { host: "127.13.157.188" port: 41595 } } }
I20260925 02:45:34.956279 27677 sys_catalog.cc:458] T 00000000000000000000000000000000 P 894f7d93b7df4cd19c802e234a6c761b [sys.catalog]: This master's current role is: LEADER
I20260925 02:45:34.956560 27725 mvcc.cc:205] Tried to move back new op lower bound from 7333086555890008064 to 7333086555838173184. Current Snapshot: MvccSnapshot[applied={T|T < 7333086555890008064}]
I20260925 02:45:34.956578 27676 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f34aeb99aad45a2a58463a851771b79 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "894f7d93b7df4cd19c802e234a6c761b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f34aeb99aad45a2a58463a851771b79" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 36279 } } peers { permanent_uuid: "894f7d93b7df4cd19c802e234a6c761b" member_type: VOTER last_known_addr { host: "127.13.157.189" port: 46813 } } peers { permanent_uuid: "2a7d13744129452a9c0cdc77249ab4ff" member_type: VOTER last_known_addr { host: "127.13.157.188" port: 41595 } } }
I20260925 02:45:34.956689 27676 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f34aeb99aad45a2a58463a851771b79 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 02:45:34.957599 27728 mvcc.cc:205] Tried to move back new op lower bound from 7333086555890008064 to 7333086555838173184. Current Snapshot: MvccSnapshot[applied={T|T < 7333086555890008064}]
I20260925 02:45:34.963997 27722 sys_catalog.cc:455] T 00000000000000000000000000000000 P 894f7d93b7df4cd19c802e234a6c761b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "894f7d93b7df4cd19c802e234a6c761b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f34aeb99aad45a2a58463a851771b79" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 36279 } } peers { permanent_uuid: "894f7d93b7df4cd19c802e234a6c761b" member_type: VOTER last_known_addr { host: "127.13.157.189" port: 46813 } } peers { permanent_uuid: "2a7d13744129452a9c0cdc77249ab4ff" member_type: VOTER last_known_addr { host: "127.13.157.188" port: 41595 } } }
I20260925 02:45:34.964123 27722 sys_catalog.cc:458] T 00000000000000000000000000000000 P 894f7d93b7df4cd19c802e234a6c761b [sys.catalog]: This master's current role is: LEADER
I20260925 02:45:34.966022 27709 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a7d13744129452a9c0cdc77249ab4ff [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "894f7d93b7df4cd19c802e234a6c761b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f34aeb99aad45a2a58463a851771b79" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 36279 } } peers { permanent_uuid: "894f7d93b7df4cd19c802e234a6c761b" member_type: VOTER last_known_addr { host: "127.13.157.189" port: 46813 } } peers { permanent_uuid: "2a7d13744129452a9c0cdc77249ab4ff" member_type: VOTER last_known_addr { host: "127.13.157.188" port: 41595 } } }
I20260925 02:45:34.966143 27709 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a7d13744129452a9c0cdc77249ab4ff [sys.catalog]: This master's current role is: FOLLOWER
I20260925 02:45:34.970068 27726 mvcc.cc:205] Tried to move back new op lower bound from 7333086555890008064 to 7333086555838173184. Current Snapshot: MvccSnapshot[applied={T|T < 7333086555890008064}]
I20260925 02:45:34.971344 27724 catalog_manager.cc:1347] Generated new cluster ID: 10660bb5a4894e2084d24a270ac2046e
I20260925 02:45:34.971417 27724 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 02:45:34.993937 27724 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 02:45:34.995191 27724 catalog_manager.cc:1504] Loading token signing keys...
I20260925 02:45:35.010984 27724 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 894f7d93b7df4cd19c802e234a6c761b: Generated new TSK 0
I20260925 02:45:35.011854 27724 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 02:45:35.896347 27701 catalog_manager.cc:1259] Loaded cluster ID: 10660bb5a4894e2084d24a270ac2046e
I20260925 02:45:35.896435 27701 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 7f34aeb99aad45a2a58463a851771b79: loading cluster ID for follower catalog manager: success
I20260925 02:45:35.897923 27720 catalog_manager.cc:1259] Loaded cluster ID: 10660bb5a4894e2084d24a270ac2046e
I20260925 02:45:35.897991 27720 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 2a7d13744129452a9c0cdc77249ab4ff: loading cluster ID for follower catalog manager: success
I20260925 02:45:35.898139 27701 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 7f34aeb99aad45a2a58463a851771b79: acquiring CA information for follower catalog manager: success
I20260925 02:45:35.899457 27701 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 7f34aeb99aad45a2a58463a851771b79: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 02:45:35.899623 27720 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 2a7d13744129452a9c0cdc77249ab4ff: acquiring CA information for follower catalog manager: success
I20260925 02:45:35.900653 27720 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 2a7d13744129452a9c0cdc77249ab4ff: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 02:45:36.078361 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 27449
I20260925 02:45:36.084643 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 27524
I20260925 02:45:36.089965 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 27599
2026-09-25T02:45:36Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (2965 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790304178325258-13942-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 02:45:36 dist-test-slave-kznw krb5kdc[27739](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 02:45:36 dist-test-slave-kznw krb5kdc[27739](info): set up 2 sockets
Sep 25 02:45:36 dist-test-slave-kznw krb5kdc[27739](info): commencing operation
krb5kdc: starting...
W20260925 02:45:37.047219 13942 mini_kdc.cc:121] Time spent starting KDC: real 0.936s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790304178325258-13942-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790304178325258-13942-0/krb5kdc/test-user.keytab.
Sep 25 02:45:37 dist-test-slave-kznw krb5kdc[27739](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304337, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T02:45:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T02:45:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.157.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.157.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.157.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab.
Entry for principal kudu/127.13.157.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab.
I20260925 02:45:37.166559 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790304178325258-13942-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790304178325258-13942-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790304178325258-13942-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.157.190:44303
--webserver_interface=127.13.157.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab
--principal=kudu/127.13.157.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:46871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.157.190:44303
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:45:37.277388 27752 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:45:37.277781 27752 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:45:37.277855 27752 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:45:37.282117 27752 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:45:37.282177 27752 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:45:37.282202 27752 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:45:37.282225 27752 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:45:37.282253 27752 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:45:37.287724 27752 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:46871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790304178325258-13942-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790304178325258-13942-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.157.190:44303
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab
--principal=kudu/127.13.157.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.157.190:44303
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.157.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27752
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790304178325258-13942-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:45:37.288825 27752 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:45:37.289885 27752 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:45:37.298460 27760 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:45:37.298532 27752 server_base.cc:1034] running on GCE node
W20260925 02:45:37.298480 27757 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:45:37.298532 27758 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:45:37.299342 27752 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:45:37.300518 27752 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:45:37.301728 27752 hybrid_clock.cc:648] HybridClock initialized: now 1790304337301708 us; error 31 us; skew 500 ppm
Sep 25 02:45:37 dist-test-slave-kznw krb5kdc[27739](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304337, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:45:37.304772 27752 init.cc:377] Logged in from keytab as kudu/127.13.157.190@KRBTEST.COM (short username kudu)
I20260925 02:45:37.305670 27752 webserver.cc:460] Webserver started at http://127.13.157.190:35397/ using document root <none> and password file <none>
I20260925 02:45:37.306208 27752 fs_manager.cc:359] Metadata directory not provided
I20260925 02:45:37.306277 27752 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:45:37.306463 27752 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:45:37.308585 27752 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790304178325258-13942-0/minicluster-data/master-0/data/instance:
uuid: "e8fb5ed518a64173b7b66d0246d34de4"
format_stamp: "Formatted at 2026-09-25 02:45:37 on dist-test-slave-kznw"
server_key: "effdb972c89f937f39c04e83bea44d23"
server_key_iv: "b22b8b089cadfecebedf5e3cab2eb7c3"
server_key_version: "encryptionkey@0"
I20260925 02:45:37.309154 27752 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790304178325258-13942-0/minicluster-data/master-0/wal/instance:
uuid: "e8fb5ed518a64173b7b66d0246d34de4"
format_stamp: "Formatted at 2026-09-25 02:45:37 on dist-test-slave-kznw"
server_key: "effdb972c89f937f39c04e83bea44d23"
server_key_iv: "b22b8b089cadfecebedf5e3cab2eb7c3"
server_key_version: "encryptionkey@0"
I20260925 02:45:37.313000 27752 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260925 02:45:37.315424 27767 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:45:37.316660 27752 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260925 02:45:37.316776 27752 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790304178325258-13942-0/minicluster-data/master-0/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790304178325258-13942-0/minicluster-data/master-0/wal
uuid: "e8fb5ed518a64173b7b66d0246d34de4"
format_stamp: "Formatted at 2026-09-25 02:45:37 on dist-test-slave-kznw"
server_key: "effdb972c89f937f39c04e83bea44d23"
server_key_iv: "b22b8b089cadfecebedf5e3cab2eb7c3"
server_key_version: "encryptionkey@0"
I20260925 02:45:37.316882 27752 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790304178325258-13942-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790304178325258-13942-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790304178325258-13942-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:45:37.334708 27752 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:45:37.337808 27752 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:45:37.338002 27752 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:45:37.344849 27752 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.190:44303
I20260925 02:45:37.344872 27819 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.190:44303 every 8 connection(s)
I20260925 02:45:37.345817 27752 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
I20260925 02:45:37.348599 27820 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:45:37.352989 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 27752
I20260925 02:45:37.353117 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790304178325258-13942-0/minicluster-data/master-0/wal/instance
I20260925 02:45:37.353405 13942 external_mini_cluster.cc:1428] Setting key c5d79358e2b5b95513ea64a9948e6709
I20260925 02:45:37.354997 27820 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8fb5ed518a64173b7b66d0246d34de4: Bootstrap starting.
I20260925 02:45:37.357467 27820 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e8fb5ed518a64173b7b66d0246d34de4: Neither blocks nor log segments found. Creating new log.
I20260925 02:45:37.358188 27820 log.cc:824] T 00000000000000000000000000000000 P e8fb5ed518a64173b7b66d0246d34de4: Log is configured to *not* fsync() on all Append() calls
I20260925 02:45:37.360074 27820 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8fb5ed518a64173b7b66d0246d34de4: No bootstrap required, opened a new log
Sep 25 02:45:37 dist-test-slave-kznw krb5kdc[27739](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304337, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.157.190@KRBTEST.COM
I20260925 02:45:37.363559 27820 raft_consensus.cc:348] T 00000000000000000000000000000000 P e8fb5ed518a64173b7b66d0246d34de4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8fb5ed518a64173b7b66d0246d34de4" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 44303 } }
I20260925 02:45:37.363787 27820 raft_consensus.cc:374] T 00000000000000000000000000000000 P e8fb5ed518a64173b7b66d0246d34de4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:45:37.363844 27820 raft_consensus.cc:729] T 00000000000000000000000000000000 P e8fb5ed518a64173b7b66d0246d34de4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8fb5ed518a64173b7b66d0246d34de4, State: Initialized, Role: FOLLOWER
I20260925 02:45:37.364248 27820 consensus_queue.cc:260] T 00000000000000000000000000000000 P e8fb5ed518a64173b7b66d0246d34de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8fb5ed518a64173b7b66d0246d34de4" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 44303 } }
I20260925 02:45:37.364377 27820 raft_consensus.cc:388] T 00000000000000000000000000000000 P e8fb5ed518a64173b7b66d0246d34de4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 02:45:37.364439 27820 raft_consensus.cc:482] T 00000000000000000000000000000000 P e8fb5ed518a64173b7b66d0246d34de4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 02:45:37.364535 27820 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e8fb5ed518a64173b7b66d0246d34de4 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:45:37.365360 27820 raft_consensus.cc:504] T 00000000000000000000000000000000 P e8fb5ed518a64173b7b66d0246d34de4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8fb5ed518a64173b7b66d0246d34de4" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 44303 } }
I20260925 02:45:37.365674 27820 leader_election.cc:302] T 00000000000000000000000000000000 P e8fb5ed518a64173b7b66d0246d34de4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e8fb5ed518a64173b7b66d0246d34de4; no voters:
I20260925 02:45:37.365975 27820 leader_election.cc:288] T 00000000000000000000000000000000 P e8fb5ed518a64173b7b66d0246d34de4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 02:45:37.366250 27825 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e8fb5ed518a64173b7b66d0246d34de4 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:45:37.366477 27825 raft_consensus.cc:686] T 00000000000000000000000000000000 P e8fb5ed518a64173b7b66d0246d34de4 [term 1 LEADER]: Becoming Leader. State: Replica: e8fb5ed518a64173b7b66d0246d34de4, State: Running, Role: LEADER
I20260925 02:45:37.366832 27825 consensus_queue.cc:237] T 00000000000000000000000000000000 P e8fb5ed518a64173b7b66d0246d34de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8fb5ed518a64173b7b66d0246d34de4" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 44303 } }
I20260925 02:45:37.367105 27823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:37.354786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43960 (local address 127.13.157.190:44303)
0925 02:45:37.355229 (+ 443us) server_negotiation.cc:206] Beginning negotiation
0925 02:45:37.355245 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0925 02:45:37.355282 (+ 37us) server_negotiation.cc:407] Connection header received
0925 02:45:37.355930 (+ 648us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:45:37.355947 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:45:37.356284 (+ 337us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:45:37.356562 (+ 278us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:45:37.357413 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:37.358531 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:37.359467 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:37.359947 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:37.360097 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:45:37.362419 (+ 2322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:45:37.362424 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:45:37.362437 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:45:37.362464 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:45:37.364105 (+ 1641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:45:37.364529 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:45:37.364535 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:45:37.364540 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:45:37.364679 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:45:37.364938 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:45:37.364941 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:45:37.364943 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:45:37.365372 (+ 429us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:45:37.365569 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0925 02:45:37.365850 (+ 281us) server_negotiation.cc:299] Negotiation successful
0925 02:45:37.366152 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301,"thread_start_us":149,"threads_started":1}
I20260925 02:45:37.367313 27820 sys_catalog.cc:564] T 00000000000000000000000000000000 P e8fb5ed518a64173b7b66d0246d34de4 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:45:37.368036 27825 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8fb5ed518a64173b7b66d0246d34de4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e8fb5ed518a64173b7b66d0246d34de4. Latest consensus state: current_term: 1 leader_uuid: "e8fb5ed518a64173b7b66d0246d34de4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8fb5ed518a64173b7b66d0246d34de4" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 44303 } } }
I20260925 02:45:37.368160 27825 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8fb5ed518a64173b7b66d0246d34de4 [sys.catalog]: This master's current role is: LEADER
I20260925 02:45:37.368443 27826 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8fb5ed518a64173b7b66d0246d34de4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e8fb5ed518a64173b7b66d0246d34de4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8fb5ed518a64173b7b66d0246d34de4" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 44303 } } }
I20260925 02:45:37.368546 27826 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8fb5ed518a64173b7b66d0246d34de4 [sys.catalog]: This master's current role is: LEADER
I20260925 02:45:37.370193 13942 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260925 02:45:37.372586 27840 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e8fb5ed518a64173b7b66d0246d34de4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 02:45:37.372663 27840 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 02:45:37.372735 27831 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:45:37.373409 27831 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:45:37.378257 27831 catalog_manager.cc:1347] Generated new cluster ID: 3028268641e44284aa7c82f26dd515e6
I20260925 02:45:37.378320 27831 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 02:45:37.386519 27831 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 02:45:37.387517 27831 catalog_manager.cc:1504] Loading token signing keys...
I20260925 02:45:37.397686 27831 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e8fb5ed518a64173b7b66d0246d34de4: Generated new TSK 0
I20260925 02:45:37.398427 27831 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 02:45:37.425954 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 27752
2026-09-25T02:45:37Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1334 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 02:45:37 dist-test-slave-kznw krb5kdc[27847](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 02:45:37 dist-test-slave-kznw krb5kdc[27847](info): set up 2 sockets
Sep 25 02:45:37 dist-test-slave-kznw krb5kdc[27847](info): commencing operation
krb5kdc: starting...
W20260925 02:45:38.386981 13942 mini_kdc.cc:121] Time spent starting KDC: real 0.942s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/krb5kdc/test-user.keytab.
Sep 25 02:45:38 dist-test-slave-kznw krb5kdc[27847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304338, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T02:45:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T02:45:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.157.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.157.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.157.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab.
Entry for principal kudu/127.13.157.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab.
I20260925 02:45:38.506557 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.157.190:34431
--webserver_interface=127.13.157.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab
--principal=kudu/127.13.157.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:46509
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.157.190:34431
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:45:38.616178 27860 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:45:38.616439 27860 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:45:38.616493 27860 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:45:38.620519 27860 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:45:38.620580 27860 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:45:38.620613 27860 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:45:38.620635 27860 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:45:38.620664 27860 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:45:38.625969 27860 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:46509
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.157.190:34431
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab
--principal=kudu/127.13.157.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.157.190:34431
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.157.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27860
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:45:38.627126 27860 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:45:38.628151 27860 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:45:38.636620 27865 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:45:38.636700 27866 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:45:38.636711 27868 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:45:38.636682 27860 server_base.cc:1034] running on GCE node
I20260925 02:45:38.637439 27860 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:45:38.638543 27860 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:45:38.639724 27860 hybrid_clock.cc:648] HybridClock initialized: now 1790304338639704 us; error 39 us; skew 500 ppm
Sep 25 02:45:38 dist-test-slave-kznw krb5kdc[27847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304338, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:45:38.642373 27860 init.cc:377] Logged in from keytab as kudu/127.13.157.190@KRBTEST.COM (short username kudu)
I20260925 02:45:38.643253 27860 webserver.cc:460] Webserver started at http://127.13.157.190:43715/ using document root <none> and password file <none>
I20260925 02:45:38.643791 27860 fs_manager.cc:359] Metadata directory not provided
I20260925 02:45:38.643849 27860 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:45:38.644006 27860 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:45:38.645982 27860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/master-0/data/instance:
uuid: "dc06e289c66345e0b45649408d6da197"
format_stamp: "Formatted at 2026-09-25 02:45:38 on dist-test-slave-kznw"
server_key: "f97de62a81434f997a2859b500722b3b"
server_key_iv: "2e8faf4b230cedbf5bcde21f4850148c"
server_key_version: "encryptionkey@0"
I20260925 02:45:38.646495 27860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/master-0/wal/instance:
uuid: "dc06e289c66345e0b45649408d6da197"
format_stamp: "Formatted at 2026-09-25 02:45:38 on dist-test-slave-kznw"
server_key: "f97de62a81434f997a2859b500722b3b"
server_key_iv: "2e8faf4b230cedbf5bcde21f4850148c"
server_key_version: "encryptionkey@0"
I20260925 02:45:38.650024 27860 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260925 02:45:38.652302 27875 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:45:38.653635 27860 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260925 02:45:38.653784 27860 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/master-0/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/master-0/wal
uuid: "dc06e289c66345e0b45649408d6da197"
format_stamp: "Formatted at 2026-09-25 02:45:38 on dist-test-slave-kznw"
server_key: "f97de62a81434f997a2859b500722b3b"
server_key_iv: "2e8faf4b230cedbf5bcde21f4850148c"
server_key_version: "encryptionkey@0"
I20260925 02:45:38.653878 27860 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:45:38.677309 27860 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:45:38.680472 27860 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:45:38.680662 27860 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:45:38.687639 27860 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.190:34431
I20260925 02:45:38.687896 27927 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.190:34431 every 8 connection(s)
I20260925 02:45:38.688609 27860 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
I20260925 02:45:38.691378 27928 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:45:38.692785 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 27860
I20260925 02:45:38.692905 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/master-0/wal/instance
I20260925 02:45:38.693225 13942 external_mini_cluster.cc:1428] Setting key d357cc00ab6965b35002739f2a580111
I20260925 02:45:38.697672 27928 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc06e289c66345e0b45649408d6da197: Bootstrap starting.
I20260925 02:45:38.700270 27928 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dc06e289c66345e0b45649408d6da197: Neither blocks nor log segments found. Creating new log.
I20260925 02:45:38.701148 27928 log.cc:824] T 00000000000000000000000000000000 P dc06e289c66345e0b45649408d6da197: Log is configured to *not* fsync() on all Append() calls
Sep 25 02:45:38 dist-test-slave-kznw krb5kdc[27847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304338, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.157.190@KRBTEST.COM
I20260925 02:45:38.703368 27928 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc06e289c66345e0b45649408d6da197: No bootstrap required, opened a new log
I20260925 02:45:38.707690 27931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:38.694670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41892 (local address 127.13.157.190:34431)
0925 02:45:38.695080 (+ 410us) server_negotiation.cc:206] Beginning negotiation
0925 02:45:38.695096 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0925 02:45:38.695132 (+ 36us) server_negotiation.cc:407] Connection header received
0925 02:45:38.695765 (+ 633us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:45:38.695782 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:45:38.696107 (+ 325us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:45:38.696387 (+ 280us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:45:38.697244 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:38.698344 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:38.699538 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:38.700010 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:38.700122 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:45:38.702682 (+ 2560us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:45:38.702688 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:45:38.702705 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:45:38.702736 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:45:38.704585 (+ 1849us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:45:38.704980 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:45:38.704986 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:45:38.704991 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:45:38.705132 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:45:38.705373 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:45:38.705376 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:45:38.705378 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:45:38.705886 (+ 508us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:45:38.706068 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0925 02:45:38.706369 (+ 301us) server_negotiation.cc:299] Negotiation successful
0925 02:45:38.706668 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":272,"thread_start_us":100,"threads_started":1}
I20260925 02:45:38.707780 27928 raft_consensus.cc:348] T 00000000000000000000000000000000 P dc06e289c66345e0b45649408d6da197 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc06e289c66345e0b45649408d6da197" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 34431 } }
I20260925 02:45:38.708037 27928 raft_consensus.cc:374] T 00000000000000000000000000000000 P dc06e289c66345e0b45649408d6da197 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:45:38.708092 27928 raft_consensus.cc:729] T 00000000000000000000000000000000 P dc06e289c66345e0b45649408d6da197 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc06e289c66345e0b45649408d6da197, State: Initialized, Role: FOLLOWER
I20260925 02:45:38.708559 27928 consensus_queue.cc:260] T 00000000000000000000000000000000 P dc06e289c66345e0b45649408d6da197 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc06e289c66345e0b45649408d6da197" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 34431 } }
I20260925 02:45:38.708730 27928 raft_consensus.cc:388] T 00000000000000000000000000000000 P dc06e289c66345e0b45649408d6da197 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 02:45:38.708798 27928 raft_consensus.cc:482] T 00000000000000000000000000000000 P dc06e289c66345e0b45649408d6da197 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 02:45:38.708874 27928 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dc06e289c66345e0b45649408d6da197 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:45:38.709721 27928 raft_consensus.cc:504] T 00000000000000000000000000000000 P dc06e289c66345e0b45649408d6da197 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc06e289c66345e0b45649408d6da197" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 34431 } }
I20260925 02:45:38.710069 27928 leader_election.cc:302] T 00000000000000000000000000000000 P dc06e289c66345e0b45649408d6da197 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dc06e289c66345e0b45649408d6da197; no voters:
I20260925 02:45:38.710361 27928 leader_election.cc:288] T 00000000000000000000000000000000 P dc06e289c66345e0b45649408d6da197 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 02:45:38.710479 27933 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dc06e289c66345e0b45649408d6da197 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:45:38.710687 27933 raft_consensus.cc:686] T 00000000000000000000000000000000 P dc06e289c66345e0b45649408d6da197 [term 1 LEADER]: Becoming Leader. State: Replica: dc06e289c66345e0b45649408d6da197, State: Running, Role: LEADER
I20260925 02:45:38.711076 27933 consensus_queue.cc:237] T 00000000000000000000000000000000 P dc06e289c66345e0b45649408d6da197 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc06e289c66345e0b45649408d6da197" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 34431 } }
I20260925 02:45:38.711540 27928 sys_catalog.cc:564] T 00000000000000000000000000000000 P dc06e289c66345e0b45649408d6da197 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:45:38.712595 27934 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc06e289c66345e0b45649408d6da197 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dc06e289c66345e0b45649408d6da197" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc06e289c66345e0b45649408d6da197" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 34431 } } }
I20260925 02:45:38.712725 27934 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc06e289c66345e0b45649408d6da197 [sys.catalog]: This master's current role is: LEADER
I20260925 02:45:38.713086 27942 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:45:38.713043 27935 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc06e289c66345e0b45649408d6da197 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dc06e289c66345e0b45649408d6da197. Latest consensus state: current_term: 1 leader_uuid: "dc06e289c66345e0b45649408d6da197" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc06e289c66345e0b45649408d6da197" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 34431 } } }
I20260925 02:45:38.713194 27935 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc06e289c66345e0b45649408d6da197 [sys.catalog]: This master's current role is: LEADER
I20260925 02:45:38.716499 27942 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:45:38.721966 27942 catalog_manager.cc:1347] Generated new cluster ID: 303bcc0b7d1a473e887055648a7ee134
I20260925 02:45:38.722035 27942 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 02:45:38.735680 27942 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 02:45:38.736683 27942 catalog_manager.cc:1504] Loading token signing keys...
I20260925 02:45:38.743636 27942 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dc06e289c66345e0b45649408d6da197: Generated new TSK 0
I20260925 02:45:38.744427 27942 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.157.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.157.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.157.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.129.keytab.
Entry for principal kudu/127.13.157.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.129.keytab.
I20260925 02:45:38.786480 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.157.129:0
--local_ip_for_outbound_sockets=127.13.157.129
--webserver_interface=127.13.157.129
--webserver_port=0
--tserver_master_addrs=127.13.157.190:34431
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.129.keytab
--principal=kudu/127.13.157.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:46509
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:45:38.901540 27954 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:45:38.901821 27954 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:45:38.901865 27954 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:45:38.905934 27954 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:45:38.905997 27954 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:45:38.906055 27954 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.157.129
I20260925 02:45:38.911374 27954 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:46509
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.129.keytab
--principal=kudu/127.13.157.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.157.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.157.129
--webserver_port=0
--tserver_master_addrs=127.13.157.190:34431
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27954
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.157.129
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:45:38.912550 27954 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:45:38.913647 27954 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:45:38.923460 27962 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:45:38.923470 27959 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:45:38.923451 27960 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:45:38.923708 27954 server_base.cc:1034] running on GCE node
I20260925 02:45:38.924149 27954 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:45:38.924875 27954 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:45:38.926064 27954 hybrid_clock.cc:648] HybridClock initialized: now 1790304338926042 us; error 33 us; skew 500 ppm
Sep 25 02:45:38 dist-test-slave-kznw krb5kdc[27847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304338, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:45:38.928967 27954 init.cc:377] Logged in from keytab as kudu/127.13.157.129@KRBTEST.COM (short username kudu)
I20260925 02:45:38.929854 27954 webserver.cc:460] Webserver started at http://127.13.157.129:40749/ using document root <none> and password file <none>
I20260925 02:45:38.930402 27954 fs_manager.cc:359] Metadata directory not provided
I20260925 02:45:38.930467 27954 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:45:38.930676 27954 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:45:38.932777 27954 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-0/data/instance:
uuid: "4ca40898617049bc9142388c6b5c8541"
format_stamp: "Formatted at 2026-09-25 02:45:38 on dist-test-slave-kznw"
server_key: "64b786325f8a3832ea4e8df017c64fcf"
server_key_iv: "d1c2e18f4d156da8822c91f076411eb6"
server_key_version: "encryptionkey@0"
I20260925 02:45:38.933347 27954 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-0/wal/instance:
uuid: "4ca40898617049bc9142388c6b5c8541"
format_stamp: "Formatted at 2026-09-25 02:45:38 on dist-test-slave-kznw"
server_key: "64b786325f8a3832ea4e8df017c64fcf"
server_key_iv: "d1c2e18f4d156da8822c91f076411eb6"
server_key_version: "encryptionkey@0"
I20260925 02:45:38.937207 27954 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260925 02:45:38.939944 27969 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:45:38.941370 27954 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 02:45:38.941500 27954 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-0/wal
uuid: "4ca40898617049bc9142388c6b5c8541"
format_stamp: "Formatted at 2026-09-25 02:45:38 on dist-test-slave-kznw"
server_key: "64b786325f8a3832ea4e8df017c64fcf"
server_key_iv: "d1c2e18f4d156da8822c91f076411eb6"
server_key_version: "encryptionkey@0"
I20260925 02:45:38.941610 27954 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:45:38.961580 27954 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:45:38.964540 27954 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:45:38.964720 27954 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:45:38.965329 27954 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:45:38.966174 27954 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:45:38.966221 27954 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:45:38.966275 27954 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:45:38.966300 27954 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:45:38.974949 27954 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.129:45983
I20260925 02:45:38.974978 28083 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.129:45983 every 8 connection(s)
I20260925 02:45:38.975850 27954 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-0/data/info.pb
I20260925 02:45:38.983771 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 27954
I20260925 02:45:38.983903 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-0/wal/instance
I20260925 02:45:38.984709 13942 external_mini_cluster.cc:1428] Setting key 4e9dac1875a01218c064a7da3dec65e5
Sep 25 02:45:38 dist-test-slave-kznw krb5kdc[27847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304338, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.129@KRBTEST.COM for kudu/127.13.157.190@KRBTEST.COM
I20260925 02:45:38.990758 27931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:38.977639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.129:42815 (local address 127.13.157.190:34431)
0925 02:45:38.977828 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0925 02:45:38.977832 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:45:38.978659 (+ 827us) server_negotiation.cc:407] Connection header received
0925 02:45:38.980043 (+ 1384us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:45:38.980046 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:45:38.980102 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:45:38.980179 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:45:38.981339 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:38.981882 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:38.983060 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:38.983303 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:38.983417 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:45:38.986697 (+ 3280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:45:38.986718 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:45:38.986721 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:45:38.986749 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:45:38.988271 (+ 1522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:45:38.988980 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:45:38.988983 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:45:38.988985 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:45:38.989076 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:45:38.989480 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:45:38.989484 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:45:38.989485 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:45:38.989709 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:45:38.989784 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0925 02:45:38.990468 (+ 684us) server_negotiation.cc:299] Negotiation successful
0925 02:45:38.990595 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
I20260925 02:45:38.992435 28086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:38.978024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:34431 (local address 127.13.157.129:42815)
0925 02:45:38.978505 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:38.978549 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0925 02:45:38.979829 (+ 1280us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:45:38.980343 (+ 514us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:45:38.980350 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:45:38.980700 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:45:38.981197 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:45:38.981211 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:45:38.982001 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:45:38.982004 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:45:38.982941 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:45:38.982953 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:45:38.983447 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:45:38.983472 (+ 25us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:45:38.983671 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:45:38.984348 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:45:38.984370 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:45:38.986509 (+ 2139us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:45:38.988401 (+ 1892us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:45:38.988407 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:45:38.988423 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:45:38.988863 (+ 440us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:45:38.989196 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:45:38.989199 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:45:38.989200 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:45:38.989377 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:45:38.989868 (+ 491us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:45:38.989874 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:45:38.990196 (+ 322us) client_negotiation.cc:769] Sending connection context
0925 02:45:38.990938 (+ 742us) client_negotiation.cc:240] Negotiation successful
0925 02:45:38.991313 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":139,"threads_started":1}
I20260925 02:45:38.994506 28084 heartbeater.cc:347] Connected to a master server at 127.13.157.190:34431
I20260925 02:45:38.994789 28084 heartbeater.cc:464] Registering TS with master...
I20260925 02:45:38.995554 28084 heartbeater.cc:511] Master 127.13.157.190:34431 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.13.157.130@KRBTEST.COM; defaulting to no policy
I20260925 02:45:38.997239 27890 ts_manager.cc:194] Registered new tserver with Master: 4ca40898617049bc9142388c6b5c8541 (127.13.157.129:45983)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.157.130@KRBTEST.COM" created.
I20260925 02:45:38.999061 27890 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.157.129@KRBTEST.COM'} at 127.13.157.129:42815
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.157.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.130.keytab.
Entry for principal kudu/127.13.157.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.130.keytab.
I20260925 02:45:39.014952 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.157.130:0
--local_ip_for_outbound_sockets=127.13.157.130
--webserver_interface=127.13.157.130
--webserver_port=0
--tserver_master_addrs=127.13.157.190:34431
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.130.keytab
--principal=kudu/127.13.157.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:46509
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:45:39.128518 28089 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:45:39.128782 28089 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:45:39.128825 28089 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:45:39.132898 28089 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:45:39.132965 28089 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:45:39.133024 28089 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.157.130
I20260925 02:45:39.138366 28089 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:46509
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.130.keytab
--principal=kudu/127.13.157.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.157.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.157.130
--webserver_port=0
--tserver_master_addrs=127.13.157.190:34431
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28089
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.157.130
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:45:39.139470 28089 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:45:39.140479 28089 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:45:39.150036 28097 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:45:39.150070 28089 server_base.cc:1034] running on GCE node
W20260925 02:45:39.150034 28094 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:45:39.150054 28095 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:45:39.150707 28089 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:45:39.151393 28089 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:45:39.152577 28089 hybrid_clock.cc:648] HybridClock initialized: now 1790304339152551 us; error 36 us; skew 500 ppm
Sep 25 02:45:39 dist-test-slave-kznw krb5kdc[27847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304339, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:45:39.155285 28089 init.cc:377] Logged in from keytab as kudu/127.13.157.130@KRBTEST.COM (short username kudu)
I20260925 02:45:39.156126 28089 webserver.cc:460] Webserver started at http://127.13.157.130:40957/ using document root <none> and password file <none>
I20260925 02:45:39.156684 28089 fs_manager.cc:359] Metadata directory not provided
I20260925 02:45:39.156745 28089 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:45:39.156929 28089 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:45:39.159098 28089 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-1/data/instance:
uuid: "143d3437e4be499fb78ad8ea64dfaaa4"
format_stamp: "Formatted at 2026-09-25 02:45:39 on dist-test-slave-kznw"
server_key: "9b52e7135c42992013322d738375c784"
server_key_iv: "c18cb6d8e6c95eb58062bb8b1be47467"
server_key_version: "encryptionkey@0"
I20260925 02:45:39.159670 28089 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-1/wal/instance:
uuid: "143d3437e4be499fb78ad8ea64dfaaa4"
format_stamp: "Formatted at 2026-09-25 02:45:39 on dist-test-slave-kznw"
server_key: "9b52e7135c42992013322d738375c784"
server_key_iv: "c18cb6d8e6c95eb58062bb8b1be47467"
server_key_version: "encryptionkey@0"
I20260925 02:45:39.163463 28089 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260925 02:45:39.165995 28105 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:45:39.167409 28089 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 02:45:39.167532 28089 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-1/wal
uuid: "143d3437e4be499fb78ad8ea64dfaaa4"
format_stamp: "Formatted at 2026-09-25 02:45:39 on dist-test-slave-kznw"
server_key: "9b52e7135c42992013322d738375c784"
server_key_iv: "c18cb6d8e6c95eb58062bb8b1be47467"
server_key_version: "encryptionkey@0"
I20260925 02:45:39.167656 28089 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:45:39.186342 28089 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:45:39.189252 28089 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:45:39.189455 28089 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:45:39.190173 28089 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:45:39.191078 28089 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:45:39.191128 28089 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:45:39.191176 28089 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:45:39.191205 28089 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:45:39.199942 28089 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.130:35399
I20260925 02:45:39.199980 28219 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.130:35399 every 8 connection(s)
I20260925 02:45:39.200939 28089 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-1/data/info.pb
I20260925 02:45:39.201887 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 28089
I20260925 02:45:39.202008 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-1/wal/instance
I20260925 02:45:39.202268 13942 external_mini_cluster.cc:1428] Setting key b178cd397668b30a39180759a95fedae
Sep 25 02:45:39 dist-test-slave-kznw krb5kdc[27847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304339, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.130@KRBTEST.COM for kudu/127.13.157.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.157.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.157.131@KRBTEST.COM" created.
I20260925 02:45:39.216812 27931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:39.202838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.130:50293 (local address 127.13.157.190:34431)
0925 02:45:39.203004 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0925 02:45:39.203008 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:45:39.204128 (+ 1120us) server_negotiation.cc:407] Connection header received
0925 02:45:39.205515 (+ 1387us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:45:39.205517 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:45:39.205562 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:45:39.205628 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:45:39.207099 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:39.207680 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:39.209402 (+ 1722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:39.209628 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:39.209710 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:45:39.212871 (+ 3161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:45:39.212875 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:45:39.212877 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:45:39.212906 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:45:39.214491 (+ 1585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:45:39.215125 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:45:39.215129 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:45:39.215131 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:45:39.215229 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:45:39.215622 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:45:39.215626 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:45:39.215628 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:45:39.215871 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:45:39.215972 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0925 02:45:39.216506 (+ 534us) server_negotiation.cc:299] Negotiation successful
0925 02:45:39.216639 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260925 02:45:39.217885 28222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:39.203378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:34431 (local address 127.13.157.130:50293)
0925 02:45:39.203977 (+ 599us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:39.204022 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0925 02:45:39.205268 (+ 1246us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:45:39.205838 (+ 570us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:45:39.205849 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:45:39.206337 (+ 488us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:45:39.206954 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:45:39.206971 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:45:39.207813 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:45:39.207817 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:45:39.209267 (+ 1450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:45:39.209280 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:45:39.209792 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:45:39.209797 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:45:39.210021 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:45:39.210650 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:45:39.210671 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:45:39.212632 (+ 1961us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:45:39.214621 (+ 1989us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:45:39.214627 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:45:39.214644 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:45:39.215009 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:45:39.215332 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:45:39.215335 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:45:39.215337 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:45:39.215498 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:45:39.215987 (+ 489us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:45:39.215995 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:45:39.216257 (+ 262us) client_negotiation.cc:769] Sending connection context
0925 02:45:39.216427 (+ 170us) client_negotiation.cc:240] Negotiation successful
0925 02:45:39.216786 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":130,"threads_started":1}
I20260925 02:45:39.219125 28220 heartbeater.cc:347] Connected to a master server at 127.13.157.190:34431
I20260925 02:45:39.219381 28220 heartbeater.cc:464] Registering TS with master...
I20260925 02:45:39.219904 28220 heartbeater.cc:511] Master 127.13.157.190:34431 requested a full tablet report, sending...
I20260925 02:45:39.220935 27890 ts_manager.cc:194] Registered new tserver with Master: 143d3437e4be499fb78ad8ea64dfaaa4 (127.13.157.130:35399)
I20260925 02:45:39.221486 27890 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.157.130@KRBTEST.COM'} at 127.13.157.130:50293
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.157.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.131.keytab.
Entry for principal kudu/127.13.157.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.131.keytab.
I20260925 02:45:39.230208 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.157.131:0
--local_ip_for_outbound_sockets=127.13.157.131
--webserver_interface=127.13.157.131
--webserver_port=0
--tserver_master_addrs=127.13.157.190:34431
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.131.keytab
--principal=kudu/127.13.157.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:46509
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:45:39.342736 28225 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:45:39.343060 28225 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:45:39.343114 28225 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:45:39.347210 28225 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:45:39.347263 28225 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:45:39.347324 28225 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.157.131
I20260925 02:45:39.352491 28225 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:46509
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.131.keytab
--principal=kudu/127.13.157.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.157.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.157.131
--webserver_port=0
--tserver_master_addrs=127.13.157.190:34431
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28225
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.157.131
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:45:39.353529 28225 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:45:39.354537 28225 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:45:39.364087 28230 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:45:39.364092 28231 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:45:39.364092 28233 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:45:39.364151 28225 server_base.cc:1034] running on GCE node
I20260925 02:45:39.364678 28225 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:45:39.365340 28225 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:45:39.366501 28225 hybrid_clock.cc:648] HybridClock initialized: now 1790304339366483 us; error 35 us; skew 500 ppm
Sep 25 02:45:39 dist-test-slave-kznw krb5kdc[27847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304339, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:45:39.372644 28225 init.cc:377] Logged in from keytab as kudu/127.13.157.131@KRBTEST.COM (short username kudu)
I20260925 02:45:39.373716 28225 webserver.cc:460] Webserver started at http://127.13.157.131:35755/ using document root <none> and password file <none>
I20260925 02:45:39.374311 28225 fs_manager.cc:359] Metadata directory not provided
I20260925 02:45:39.374382 28225 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:45:39.374591 28225 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:45:39.376748 28225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-2/data/instance:
uuid: "32c471b5298c474ebe1515814e5e6f35"
format_stamp: "Formatted at 2026-09-25 02:45:39 on dist-test-slave-kznw"
server_key: "97d2bc30a81a3deef3cda089102dc495"
server_key_iv: "e5cedfb7525abc455d8d832ac3cd7571"
server_key_version: "encryptionkey@0"
I20260925 02:45:39.377300 28225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-2/wal/instance:
uuid: "32c471b5298c474ebe1515814e5e6f35"
format_stamp: "Formatted at 2026-09-25 02:45:39 on dist-test-slave-kznw"
server_key: "97d2bc30a81a3deef3cda089102dc495"
server_key_iv: "e5cedfb7525abc455d8d832ac3cd7571"
server_key_version: "encryptionkey@0"
I20260925 02:45:39.381023 28225 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260925 02:45:39.383575 28241 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:45:39.384866 28225 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260925 02:45:39.384974 28225 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-2/wal
uuid: "32c471b5298c474ebe1515814e5e6f35"
format_stamp: "Formatted at 2026-09-25 02:45:39 on dist-test-slave-kznw"
server_key: "97d2bc30a81a3deef3cda089102dc495"
server_key_iv: "e5cedfb7525abc455d8d832ac3cd7571"
server_key_version: "encryptionkey@0"
I20260925 02:45:39.385072 28225 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:45:39.399128 28225 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:45:39.402278 28225 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:45:39.402458 28225 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:45:39.403105 28225 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:45:39.403936 28225 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:45:39.403985 28225 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:45:39.404037 28225 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:45:39.404067 28225 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:45:39.412490 28225 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.131:40183
I20260925 02:45:39.412619 28354 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.131:40183 every 8 connection(s)
I20260925 02:45:39.413420 28225 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-2/data/info.pb
I20260925 02:45:39.416488 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 28225
I20260925 02:45:39.416620 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-2/wal/instance
I20260925 02:45:39.416880 13942 external_mini_cluster.cc:1428] Setting key bdf8961a823017c4d9e78aa33a07eebf
Sep 25 02:45:39 dist-test-slave-kznw krb5kdc[27847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304339, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.131@KRBTEST.COM for kudu/127.13.157.190@KRBTEST.COM
I20260925 02:45:39.426530 27931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:39.415267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.131:54107 (local address 127.13.157.190:34431)
0925 02:45:39.415406 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0925 02:45:39.415411 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:45:39.416336 (+ 925us) server_negotiation.cc:407] Connection header received
0925 02:45:39.417834 (+ 1498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:45:39.417837 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:45:39.417892 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:45:39.417966 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:45:39.419141 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:39.419685 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:39.420806 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:39.421025 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:39.421111 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:45:39.423386 (+ 2275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:45:39.423389 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:45:39.423391 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:45:39.423416 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:45:39.424591 (+ 1175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:45:39.425096 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:45:39.425099 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:45:39.425100 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:45:39.425174 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:45:39.425475 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:45:39.425479 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:45:39.425481 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:45:39.425656 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:45:39.425743 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0925 02:45:39.426239 (+ 496us) server_negotiation.cc:299] Negotiation successful
0925 02:45:39.426363 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260925 02:45:39.427356 28357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:39.415694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:34431 (local address 127.13.157.131:54107)
0925 02:45:39.416178 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:39.416221 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0925 02:45:39.417593 (+ 1372us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:45:39.418133 (+ 540us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:45:39.418140 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:45:39.418498 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:45:39.419015 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:45:39.419031 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:45:39.419806 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:45:39.419809 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:45:39.420684 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:45:39.420693 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:45:39.421160 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:45:39.421164 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:45:39.421323 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:45:39.421831 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:45:39.421846 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:45:39.423234 (+ 1388us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:45:39.424695 (+ 1461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:45:39.424701 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:45:39.424717 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:45:39.424989 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:45:39.425256 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:45:39.425259 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:45:39.425261 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:45:39.425392 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:45:39.425761 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:45:39.425767 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:45:39.426007 (+ 240us) client_negotiation.cc:769] Sending connection context
0925 02:45:39.426175 (+ 168us) client_negotiation.cc:240] Negotiation successful
0925 02:45:39.426497 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":113,"threads_started":1}
I20260925 02:45:39.428467 28355 heartbeater.cc:347] Connected to a master server at 127.13.157.190:34431
I20260925 02:45:39.428678 28355 heartbeater.cc:464] Registering TS with master...
I20260925 02:45:39.429132 28355 heartbeater.cc:511] Master 127.13.157.190:34431 requested a full tablet report, sending...
I20260925 02:45:39.430299 27890 ts_manager.cc:194] Registered new tserver with Master: 32c471b5298c474ebe1515814e5e6f35 (127.13.157.131:40183)
I20260925 02:45:39.430802 13942 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 02:45:39.430887 27890 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.157.131@KRBTEST.COM'} at 127.13.157.131:54107
I20260925 02:45:39.439850 27931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:39.432879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41896 (local address 127.13.157.190:34431)
0925 02:45:39.433011 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0925 02:45:39.433015 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:45:39.433227 (+ 212us) server_negotiation.cc:407] Connection header received
0925 02:45:39.433355 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:45:39.433357 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:45:39.433396 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:45:39.433460 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:45:39.434131 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:39.434630 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:39.435485 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:39.435722 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:39.435800 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:45:39.437212 (+ 1412us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:45:39.437217 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:45:39.437220 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:45:39.437241 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:45:39.438298 (+ 1057us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:45:39.438669 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:45:39.438673 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:45:39.438674 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:45:39.438746 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:45:39.439017 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:45:39.439020 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:45:39.439022 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:45:39.439228 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:45:39.439359 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0925 02:45:39.439586 (+ 227us) server_negotiation.cc:299] Negotiation successful
0925 02:45:39.439701 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260925 02:45:39.442402 27890 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41896:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 02:45:39.445407 27890 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 02:45:39.459367 28367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:39.453550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.130:35399 (local address 127.0.0.1:37360)
0925 02:45:39.454031 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:39.454078 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0925 02:45:39.454211 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:45:39.454778 (+ 567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:45:39.454785 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:45:39.454811 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:45:39.454993 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:45:39.455002 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:45:39.456451 (+ 1449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:45:39.456453 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:45:39.457570 (+ 1117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:45:39.457580 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:45:39.458784 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:45:39.458788 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:45:39.458893 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:45:39.458961 (+ 68us) client_negotiation.cc:769] Sending connection context
0925 02:45:39.459073 (+ 112us) client_negotiation.cc:240] Negotiation successful
0925 02:45:39.459200 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"mutex_wait_us":44,"thread_start_us":86,"threads_started":1}
I20260925 02:45:39.459371 28368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:39.453680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37360 (local address 127.13.157.130:35399)
0925 02:45:39.454241 (+ 561us) server_negotiation.cc:206] Beginning negotiation
0925 02:45:39.454255 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0925 02:45:39.454305 (+ 50us) server_negotiation.cc:407] Connection header received
0925 02:45:39.454382 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:45:39.454388 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:45:39.454595 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:45:39.455148 (+ 553us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:45:39.455314 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:39.456311 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:39.457861 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:39.458671 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:39.458801 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:45:39.458957 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0925 02:45:39.459105 (+ 148us) server_negotiation.cc:299] Negotiation successful
0925 02:45:39.459203 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":104,"threads_started":1}
I20260925 02:45:39.459914 28366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:39.453664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.129:45983 (local address 127.0.0.1:60164)
0925 02:45:39.454237 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:39.454251 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 02:45:39.454332 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:45:39.455634 (+ 1302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:45:39.455637 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:45:39.455653 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:45:39.455748 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:45:39.455754 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:45:39.457134 (+ 1380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:45:39.457137 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:45:39.458331 (+ 1194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:45:39.458341 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:45:39.459519 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:45:39.459523 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:45:39.459617 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:45:39.459653 (+ 36us) client_negotiation.cc:769] Sending connection context
0925 02:45:39.459720 (+ 67us) client_negotiation.cc:240] Negotiation successful
0925 02:45:39.459786 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":513,"thread_start_us":111,"threads_started":1}
I20260925 02:45:39.460086 28371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:39.454734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60164 (local address 127.13.157.129:45983)
0925 02:45:39.455039 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0925 02:45:39.455055 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0925 02:45:39.455077 (+ 22us) server_negotiation.cc:407] Connection header received
0925 02:45:39.455161 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:45:39.455166 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:45:39.455355 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:45:39.455492 (+ 137us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:45:39.455866 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:39.457007 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:39.458439 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:39.459215 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:39.459335 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:45:39.459455 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0925 02:45:39.459755 (+ 300us) server_negotiation.cc:299] Negotiation successful
0925 02:45:39.459850 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":213,"thread_start_us":112,"threads_started":1}
I20260925 02:45:39.460743 28369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:39.454430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.131:40183 (local address 127.0.0.1:51428)
0925 02:45:39.454642 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:39.454655 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 02:45:39.454735 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:45:39.455829 (+ 1094us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:45:39.455832 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:45:39.455848 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:45:39.455936 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:45:39.455942 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:45:39.457292 (+ 1350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:45:39.457296 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:45:39.458441 (+ 1145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:45:39.458450 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:45:39.460395 (+ 1945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:45:39.460399 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:45:39.460483 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:45:39.460506 (+ 23us) client_negotiation.cc:769] Sending connection context
0925 02:45:39.460557 (+ 51us) client_negotiation.cc:240] Negotiation successful
0925 02:45:39.460617 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"mutex_wait_us":6,"thread_start_us":95,"threads_started":1}
I20260925 02:45:39.461411 28370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:39.454597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51428 (local address 127.13.157.131:40183)
0925 02:45:39.455413 (+ 816us) server_negotiation.cc:206] Beginning negotiation
0925 02:45:39.455424 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 02:45:39.455445 (+ 21us) server_negotiation.cc:407] Connection header received
0925 02:45:39.455514 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:45:39.455518 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:45:39.455696 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:45:39.455843 (+ 147us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:45:39.456046 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:39.457142 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:39.458557 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:39.459378 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:39.461022 (+ 1644us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:45:39.461136 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0925 02:45:39.461182 (+ 46us) server_negotiation.cc:299] Negotiation successful
0925 02:45:39.461277 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":718,"thread_start_us":140,"threads_started":1}
I20260925 02:45:39.462241 28153 tablet_service.cc:1467] Processing CreateTablet for tablet a55352a03da243de9a8ff1f62a95cc35 (DEFAULT_TABLE table=test-table [id=a34a880214154660a9c319ec9f3f53ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:45:39.462888 28017 tablet_service.cc:1467] Processing CreateTablet for tablet a55352a03da243de9a8ff1f62a95cc35 (DEFAULT_TABLE table=test-table [id=a34a880214154660a9c319ec9f3f53ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:45:39.463442 28153 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a55352a03da243de9a8ff1f62a95cc35. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:45:39.463862 28017 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a55352a03da243de9a8ff1f62a95cc35. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:45:39.464056 28289 tablet_service.cc:1467] Processing CreateTablet for tablet a55352a03da243de9a8ff1f62a95cc35 (DEFAULT_TABLE table=test-table [id=a34a880214154660a9c319ec9f3f53ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:45:39.465044 28289 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a55352a03da243de9a8ff1f62a95cc35. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:45:39.469754 28372 tablet_bootstrap.cc:492] T a55352a03da243de9a8ff1f62a95cc35 P 4ca40898617049bc9142388c6b5c8541: Bootstrap starting.
I20260925 02:45:39.471164 28373 tablet_bootstrap.cc:492] T a55352a03da243de9a8ff1f62a95cc35 P 143d3437e4be499fb78ad8ea64dfaaa4: Bootstrap starting.
I20260925 02:45:39.472291 28374 tablet_bootstrap.cc:492] T a55352a03da243de9a8ff1f62a95cc35 P 32c471b5298c474ebe1515814e5e6f35: Bootstrap starting.
I20260925 02:45:39.472644 28372 tablet_bootstrap.cc:654] T a55352a03da243de9a8ff1f62a95cc35 P 4ca40898617049bc9142388c6b5c8541: Neither blocks nor log segments found. Creating new log.
I20260925 02:45:39.473654 28373 tablet_bootstrap.cc:654] T a55352a03da243de9a8ff1f62a95cc35 P 143d3437e4be499fb78ad8ea64dfaaa4: Neither blocks nor log segments found. Creating new log.
I20260925 02:45:39.473999 28374 tablet_bootstrap.cc:654] T a55352a03da243de9a8ff1f62a95cc35 P 32c471b5298c474ebe1515814e5e6f35: Neither blocks nor log segments found. Creating new log.
I20260925 02:45:39.474036 28372 log.cc:824] T a55352a03da243de9a8ff1f62a95cc35 P 4ca40898617049bc9142388c6b5c8541: Log is configured to *not* fsync() on all Append() calls
I20260925 02:45:39.474452 28373 log.cc:824] T a55352a03da243de9a8ff1f62a95cc35 P 143d3437e4be499fb78ad8ea64dfaaa4: Log is configured to *not* fsync() on all Append() calls
I20260925 02:45:39.475111 28374 log.cc:824] T a55352a03da243de9a8ff1f62a95cc35 P 32c471b5298c474ebe1515814e5e6f35: Log is configured to *not* fsync() on all Append() calls
I20260925 02:45:39.476054 28372 tablet_bootstrap.cc:492] T a55352a03da243de9a8ff1f62a95cc35 P 4ca40898617049bc9142388c6b5c8541: No bootstrap required, opened a new log
I20260925 02:45:39.476226 28372 ts_tablet_manager.cc:1397] T a55352a03da243de9a8ff1f62a95cc35 P 4ca40898617049bc9142388c6b5c8541: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260925 02:45:39.476717 28374 tablet_bootstrap.cc:492] T a55352a03da243de9a8ff1f62a95cc35 P 32c471b5298c474ebe1515814e5e6f35: No bootstrap required, opened a new log
I20260925 02:45:39.476887 28374 ts_tablet_manager.cc:1397] T a55352a03da243de9a8ff1f62a95cc35 P 32c471b5298c474ebe1515814e5e6f35: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260925 02:45:39.479264 28372 raft_consensus.cc:348] T a55352a03da243de9a8ff1f62a95cc35 P 4ca40898617049bc9142388c6b5c8541 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "143d3437e4be499fb78ad8ea64dfaaa4" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 35399 } } peers { permanent_uuid: "4ca40898617049bc9142388c6b5c8541" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 45983 } } peers { permanent_uuid: "32c471b5298c474ebe1515814e5e6f35" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 40183 } }
I20260925 02:45:39.479493 28372 raft_consensus.cc:374] T a55352a03da243de9a8ff1f62a95cc35 P 4ca40898617049bc9142388c6b5c8541 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:45:39.479549 28372 raft_consensus.cc:729] T a55352a03da243de9a8ff1f62a95cc35 P 4ca40898617049bc9142388c6b5c8541 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ca40898617049bc9142388c6b5c8541, State: Initialized, Role: FOLLOWER
I20260925 02:45:39.479714 28374 raft_consensus.cc:348] T a55352a03da243de9a8ff1f62a95cc35 P 32c471b5298c474ebe1515814e5e6f35 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "143d3437e4be499fb78ad8ea64dfaaa4" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 35399 } } peers { permanent_uuid: "4ca40898617049bc9142388c6b5c8541" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 45983 } } peers { permanent_uuid: "32c471b5298c474ebe1515814e5e6f35" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 40183 } }
I20260925 02:45:39.479907 28374 raft_consensus.cc:374] T a55352a03da243de9a8ff1f62a95cc35 P 32c471b5298c474ebe1515814e5e6f35 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:45:39.479954 28374 raft_consensus.cc:729] T a55352a03da243de9a8ff1f62a95cc35 P 32c471b5298c474ebe1515814e5e6f35 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32c471b5298c474ebe1515814e5e6f35, State: Initialized, Role: FOLLOWER
I20260925 02:45:39.479962 28372 consensus_queue.cc:260] T a55352a03da243de9a8ff1f62a95cc35 P 4ca40898617049bc9142388c6b5c8541 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "143d3437e4be499fb78ad8ea64dfaaa4" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 35399 } } peers { permanent_uuid: "4ca40898617049bc9142388c6b5c8541" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 45983 } } peers { permanent_uuid: "32c471b5298c474ebe1515814e5e6f35" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 40183 } }
I20260925 02:45:39.480347 28374 consensus_queue.cc:260] T a55352a03da243de9a8ff1f62a95cc35 P 32c471b5298c474ebe1515814e5e6f35 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "143d3437e4be499fb78ad8ea64dfaaa4" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 35399 } } peers { permanent_uuid: "4ca40898617049bc9142388c6b5c8541" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 45983 } } peers { permanent_uuid: "32c471b5298c474ebe1515814e5e6f35" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 40183 } }
I20260925 02:45:39.480625 28084 heartbeater.cc:503] Master 127.13.157.190:34431 was elected leader, sending a full tablet report...
I20260925 02:45:39.481031 28355 heartbeater.cc:503] Master 127.13.157.190:34431 was elected leader, sending a full tablet report...
I20260925 02:45:39.481278 28374 ts_tablet_manager.cc:1428] T a55352a03da243de9a8ff1f62a95cc35 P 32c471b5298c474ebe1515814e5e6f35: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20260925 02:45:39.481937 28372 ts_tablet_manager.cc:1428] T a55352a03da243de9a8ff1f62a95cc35 P 4ca40898617049bc9142388c6b5c8541: Time spent starting tablet: real 0.006s user 0.002s sys 0.003s
I20260925 02:45:39.483544 28373 tablet_bootstrap.cc:492] T a55352a03da243de9a8ff1f62a95cc35 P 143d3437e4be499fb78ad8ea64dfaaa4: No bootstrap required, opened a new log
I20260925 02:45:39.483794 28373 ts_tablet_manager.cc:1397] T a55352a03da243de9a8ff1f62a95cc35 P 143d3437e4be499fb78ad8ea64dfaaa4: Time spent bootstrapping tablet: real 0.013s user 0.005s sys 0.000s
I20260925 02:45:39.486820 28373 raft_consensus.cc:348] T a55352a03da243de9a8ff1f62a95cc35 P 143d3437e4be499fb78ad8ea64dfaaa4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "143d3437e4be499fb78ad8ea64dfaaa4" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 35399 } } peers { permanent_uuid: "4ca40898617049bc9142388c6b5c8541" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 45983 } } peers { permanent_uuid: "32c471b5298c474ebe1515814e5e6f35" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 40183 } }
I20260925 02:45:39.487066 28373 raft_consensus.cc:374] T a55352a03da243de9a8ff1f62a95cc35 P 143d3437e4be499fb78ad8ea64dfaaa4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:45:39.487135 28373 raft_consensus.cc:729] T a55352a03da243de9a8ff1f62a95cc35 P 143d3437e4be499fb78ad8ea64dfaaa4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 143d3437e4be499fb78ad8ea64dfaaa4, State: Initialized, Role: FOLLOWER
I20260925 02:45:39.487504 28373 consensus_queue.cc:260] T a55352a03da243de9a8ff1f62a95cc35 P 143d3437e4be499fb78ad8ea64dfaaa4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "143d3437e4be499fb78ad8ea64dfaaa4" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 35399 } } peers { permanent_uuid: "4ca40898617049bc9142388c6b5c8541" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 45983 } } peers { permanent_uuid: "32c471b5298c474ebe1515814e5e6f35" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 40183 } }
I20260925 02:45:39.488111 28220 heartbeater.cc:503] Master 127.13.157.190:34431 was elected leader, sending a full tablet report...
I20260925 02:45:39.488299 28373 ts_tablet_manager.cc:1428] T a55352a03da243de9a8ff1f62a95cc35 P 143d3437e4be499fb78ad8ea64dfaaa4: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260925 02:45:39.488581 28380 raft_consensus.cc:482] T a55352a03da243de9a8ff1f62a95cc35 P 143d3437e4be499fb78ad8ea64dfaaa4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 02:45:39.488713 28380 raft_consensus.cc:504] T a55352a03da243de9a8ff1f62a95cc35 P 143d3437e4be499fb78ad8ea64dfaaa4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "143d3437e4be499fb78ad8ea64dfaaa4" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 35399 } } peers { permanent_uuid: "4ca40898617049bc9142388c6b5c8541" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 45983 } } peers { permanent_uuid: "32c471b5298c474ebe1515814e5e6f35" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 40183 } }
I20260925 02:45:39.489786 28380 leader_election.cc:288] T a55352a03da243de9a8ff1f62a95cc35 P 143d3437e4be499fb78ad8ea64dfaaa4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ca40898617049bc9142388c6b5c8541 (127.13.157.129:45983), 32c471b5298c474ebe1515814e5e6f35 (127.13.157.131:40183)
I20260925 02:45:39.494299 28222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:39.489931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.129:45983 (local address 127.13.157.130:38039)
0925 02:45:39.490041 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:39.490055 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 02:45:39.490130 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:45:39.490661 (+ 531us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:45:39.490663 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:45:39.490690 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:45:39.490870 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:45:39.490875 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:45:39.491963 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:45:39.491967 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:45:39.493158 (+ 1191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:45:39.493168 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:45:39.493982 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:45:39.493985 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:45:39.494062 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:45:39.494081 (+ 19us) client_negotiation.cc:769] Sending connection context
0925 02:45:39.494132 (+ 51us) client_negotiation.cc:240] Negotiation successful
0925 02:45:39.494191 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31,"mutex_wait_us":138,"spinlock_wait_cycles":21248}
I20260925 02:45:39.494306 28371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:39.490256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.130:38039 (local address 127.13.157.129:45983)
0925 02:45:39.490373 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0925 02:45:39.490377 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:45:39.490392 (+ 15us) server_negotiation.cc:407] Connection header received
0925 02:45:39.490440 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:45:39.490443 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:45:39.490489 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:45:39.490560 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:45:39.491000 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:39.491786 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:39.493295 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:39.493882 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:39.494001 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:45:39.494051 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0925 02:45:39.494154 (+ 103us) server_negotiation.cc:299] Negotiation successful
0925 02:45:39.494213 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260925 02:45:39.494820 28381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:39.489941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.131:40183 (local address 127.13.157.130:60431)
0925 02:45:39.490234 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:39.490246 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 02:45:39.490318 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:45:39.490673 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:45:39.490675 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:45:39.490706 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:45:39.490874 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:45:39.490882 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:45:39.491941 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:45:39.491944 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:45:39.493132 (+ 1188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:45:39.493141 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:45:39.494442 (+ 1301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:45:39.494446 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:45:39.494533 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:45:39.494555 (+ 22us) client_negotiation.cc:769] Sending connection context
0925 02:45:39.494623 (+ 68us) client_negotiation.cc:240] Negotiation successful
0925 02:45:39.494682 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236,"mutex_wait_us":128,"spinlock_wait_cycles":9984,"thread_start_us":109,"threads_started":1}
I20260925 02:45:39.494825 28370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:39.490306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.130:60431 (local address 127.13.157.131:40183)
0925 02:45:39.490419 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0925 02:45:39.490423 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:45:39.490435 (+ 12us) server_negotiation.cc:407] Connection header received
0925 02:45:39.490474 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:45:39.490476 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:45:39.490509 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:45:39.490568 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:45:39.491051 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:39.491812 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:39.493245 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:39.493882 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:39.494001 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:45:39.494051 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0925 02:45:39.494654 (+ 603us) server_negotiation.cc:299] Negotiation successful
0925 02:45:39.494738 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260925 02:45:39.494966 28037 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a55352a03da243de9a8ff1f62a95cc35" candidate_uuid: "143d3437e4be499fb78ad8ea64dfaaa4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ca40898617049bc9142388c6b5c8541" is_pre_election: true
I20260925 02:45:39.495237 28037 raft_consensus.cc:2454] T a55352a03da243de9a8ff1f62a95cc35 P 4ca40898617049bc9142388c6b5c8541 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 143d3437e4be499fb78ad8ea64dfaaa4 in term 0.
I20260925 02:45:39.495549 28309 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a55352a03da243de9a8ff1f62a95cc35" candidate_uuid: "143d3437e4be499fb78ad8ea64dfaaa4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32c471b5298c474ebe1515814e5e6f35" is_pre_election: true
I20260925 02:45:39.495707 28107 leader_election.cc:302] T a55352a03da243de9a8ff1f62a95cc35 P 143d3437e4be499fb78ad8ea64dfaaa4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 143d3437e4be499fb78ad8ea64dfaaa4, 4ca40898617049bc9142388c6b5c8541; no voters:
I20260925 02:45:39.495844 28309 raft_consensus.cc:2454] T a55352a03da243de9a8ff1f62a95cc35 P 32c471b5298c474ebe1515814e5e6f35 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 143d3437e4be499fb78ad8ea64dfaaa4 in term 0.
I20260925 02:45:39.495965 28380 raft_consensus.cc:2781] T a55352a03da243de9a8ff1f62a95cc35 P 143d3437e4be499fb78ad8ea64dfaaa4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 02:45:39.496052 28380 raft_consensus.cc:482] T a55352a03da243de9a8ff1f62a95cc35 P 143d3437e4be499fb78ad8ea64dfaaa4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 02:45:39.496100 28380 raft_consensus.cc:3037] T a55352a03da243de9a8ff1f62a95cc35 P 143d3437e4be499fb78ad8ea64dfaaa4 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:45:39.497169 28380 raft_consensus.cc:504] T a55352a03da243de9a8ff1f62a95cc35 P 143d3437e4be499fb78ad8ea64dfaaa4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "143d3437e4be499fb78ad8ea64dfaaa4" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 35399 } } peers { permanent_uuid: "4ca40898617049bc9142388c6b5c8541" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 45983 } } peers { permanent_uuid: "32c471b5298c474ebe1515814e5e6f35" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 40183 } }
I20260925 02:45:39.497478 28380 leader_election.cc:288] T a55352a03da243de9a8ff1f62a95cc35 P 143d3437e4be499fb78ad8ea64dfaaa4 [CANDIDATE]: Term 1 election: Requested vote from peers 4ca40898617049bc9142388c6b5c8541 (127.13.157.129:45983), 32c471b5298c474ebe1515814e5e6f35 (127.13.157.131:40183)
I20260925 02:45:39.497820 28309 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a55352a03da243de9a8ff1f62a95cc35" candidate_uuid: "143d3437e4be499fb78ad8ea64dfaaa4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32c471b5298c474ebe1515814e5e6f35"
I20260925 02:45:39.497833 28037 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a55352a03da243de9a8ff1f62a95cc35" candidate_uuid: "143d3437e4be499fb78ad8ea64dfaaa4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ca40898617049bc9142388c6b5c8541"
I20260925 02:45:39.497926 28309 raft_consensus.cc:3037] T a55352a03da243de9a8ff1f62a95cc35 P 32c471b5298c474ebe1515814e5e6f35 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:45:39.497951 28037 raft_consensus.cc:3037] T a55352a03da243de9a8ff1f62a95cc35 P 4ca40898617049bc9142388c6b5c8541 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:45:39.498956 28037 raft_consensus.cc:2454] T a55352a03da243de9a8ff1f62a95cc35 P 4ca40898617049bc9142388c6b5c8541 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 143d3437e4be499fb78ad8ea64dfaaa4 in term 1.
I20260925 02:45:39.499047 28309 raft_consensus.cc:2454] T a55352a03da243de9a8ff1f62a95cc35 P 32c471b5298c474ebe1515814e5e6f35 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 143d3437e4be499fb78ad8ea64dfaaa4 in term 1.
I20260925 02:45:39.499243 28107 leader_election.cc:302] T a55352a03da243de9a8ff1f62a95cc35 P 143d3437e4be499fb78ad8ea64dfaaa4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 143d3437e4be499fb78ad8ea64dfaaa4, 4ca40898617049bc9142388c6b5c8541; no voters:
I20260925 02:45:39.499464 28380 raft_consensus.cc:2781] T a55352a03da243de9a8ff1f62a95cc35 P 143d3437e4be499fb78ad8ea64dfaaa4 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:45:39.499682 28380 raft_consensus.cc:686] T a55352a03da243de9a8ff1f62a95cc35 P 143d3437e4be499fb78ad8ea64dfaaa4 [term 1 LEADER]: Becoming Leader. State: Replica: 143d3437e4be499fb78ad8ea64dfaaa4, State: Running, Role: LEADER
I20260925 02:45:39.500001 28380 consensus_queue.cc:237] T a55352a03da243de9a8ff1f62a95cc35 P 143d3437e4be499fb78ad8ea64dfaaa4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "143d3437e4be499fb78ad8ea64dfaaa4" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 35399 } } peers { permanent_uuid: "4ca40898617049bc9142388c6b5c8541" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 45983 } } peers { permanent_uuid: "32c471b5298c474ebe1515814e5e6f35" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 40183 } }
I20260925 02:45:39.503094 27890 catalog_manager.cc:5495] T a55352a03da243de9a8ff1f62a95cc35 P 143d3437e4be499fb78ad8ea64dfaaa4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 143d3437e4be499fb78ad8ea64dfaaa4 (127.13.157.130). New cstate: current_term: 1 leader_uuid: "143d3437e4be499fb78ad8ea64dfaaa4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "143d3437e4be499fb78ad8ea64dfaaa4" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 35399 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4ca40898617049bc9142388c6b5c8541" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 45983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "32c471b5298c474ebe1515814e5e6f35" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 40183 } health_report { overall_health: UNKNOWN } } }
I20260925 02:45:39.524423 28368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:39.520916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37372 (local address 127.13.157.130:35399)
0925 02:45:39.521041 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0925 02:45:39.521044 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 02:45:39.521058 (+ 14us) server_negotiation.cc:407] Connection header received
0925 02:45:39.521192 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:45:39.521194 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:45:39.521253 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:45:39.521344 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 02:45:39.521552 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:39.522065 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:39.522958 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:39.523201 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:39.523287 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:45:39.523609 (+ 322us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 02:45:39.524051 (+ 442us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 02:45:39.524166 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0925 02:45:39.524254 (+ 88us) server_negotiation.cc:299] Negotiation successful
0925 02:45:39.524307 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260925 02:45:39.530732 28309 raft_consensus.cc:1261] T a55352a03da243de9a8ff1f62a95cc35 P 32c471b5298c474ebe1515814e5e6f35 [term 1 FOLLOWER]: Refusing update from remote peer 143d3437e4be499fb78ad8ea64dfaaa4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:45:39.530750 28037 raft_consensus.cc:1261] T a55352a03da243de9a8ff1f62a95cc35 P 4ca40898617049bc9142388c6b5c8541 [term 1 FOLLOWER]: Refusing update from remote peer 143d3437e4be499fb78ad8ea64dfaaa4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:45:39.531487 28382 consensus_queue.cc:1035] T a55352a03da243de9a8ff1f62a95cc35 P 143d3437e4be499fb78ad8ea64dfaaa4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ca40898617049bc9142388c6b5c8541" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 45983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:45:39.531675 28380 consensus_queue.cc:1035] T a55352a03da243de9a8ff1f62a95cc35 P 143d3437e4be499fb78ad8ea64dfaaa4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32c471b5298c474ebe1515814e5e6f35" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 40183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:45:39.540266 28386 mvcc.cc:205] Tried to move back new op lower bound from 7333086574710026240 to 7333086574594576384. Current Snapshot: MvccSnapshot[applied={T|T < 7333086574710026240}]
I20260925 02:45:39.543160 28387 mvcc.cc:205] Tried to move back new op lower bound from 7333086574710026240 to 7333086574594576384. Current Snapshot: MvccSnapshot[applied={T|T < 7333086574710026240}]
I20260925 02:45:39.546845 27892 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41896:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 02:45:39.547040 27892 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41896:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 02:45:39.549353 27892 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dc06e289c66345e0b45649408d6da197: Sending DeleteTablet for 3 replicas of tablet a55352a03da243de9a8ff1f62a95cc35
I20260925 02:45:39.550160 28153 tablet_service.cc:1514] Processing DeleteTablet for tablet a55352a03da243de9a8ff1f62a95cc35 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:45:39 UTC) from {username='kudu', principal='kudu/127.13.157.190@KRBTEST.COM'} at 127.0.0.1:37360
I20260925 02:45:39.550173 28017 tablet_service.cc:1514] Processing DeleteTablet for tablet a55352a03da243de9a8ff1f62a95cc35 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:45:39 UTC) from {username='kudu', principal='kudu/127.13.157.190@KRBTEST.COM'} at 127.0.0.1:60164
I20260925 02:45:39.550371 28289 tablet_service.cc:1514] Processing DeleteTablet for tablet a55352a03da243de9a8ff1f62a95cc35 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:45:39 UTC) from {username='kudu', principal='kudu/127.13.157.190@KRBTEST.COM'} at 127.0.0.1:51428
I20260925 02:45:39.550868 28396 tablet_replica.cc:330] T a55352a03da243de9a8ff1f62a95cc35 P 32c471b5298c474ebe1515814e5e6f35: stopping tablet replica
I20260925 02:45:39.551002 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 27954
I20260925 02:45:39.551007 28395 tablet_replica.cc:330] T a55352a03da243de9a8ff1f62a95cc35 P 143d3437e4be499fb78ad8ea64dfaaa4: stopping tablet replica
I20260925 02:45:39.552080 28395 raft_consensus.cc:2229] T a55352a03da243de9a8ff1f62a95cc35 P 143d3437e4be499fb78ad8ea64dfaaa4 [term 1 LEADER]: Raft consensus shutting down.
I20260925 02:45:39.552465 28395 raft_consensus.cc:2258] T a55352a03da243de9a8ff1f62a95cc35 P 143d3437e4be499fb78ad8ea64dfaaa4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:45:39.554068 28395 ts_tablet_manager.cc:1905] T a55352a03da243de9a8ff1f62a95cc35 P 143d3437e4be499fb78ad8ea64dfaaa4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:45:39.555639 28396 raft_consensus.cc:2229] T a55352a03da243de9a8ff1f62a95cc35 P 32c471b5298c474ebe1515814e5e6f35 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 02:45:39.555994 28396 raft_consensus.cc:2258] T a55352a03da243de9a8ff1f62a95cc35 P 32c471b5298c474ebe1515814e5e6f35 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:45:39.557029 28395 ts_tablet_manager.cc:1918] T a55352a03da243de9a8ff1f62a95cc35 P 143d3437e4be499fb78ad8ea64dfaaa4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 02:45:39.557155 28395 log.cc:1196] T a55352a03da243de9a8ff1f62a95cc35 P 143d3437e4be499fb78ad8ea64dfaaa4: Deleting WAL directory at /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-1/wal/wals/a55352a03da243de9a8ff1f62a95cc35
I20260925 02:45:39.557471 28395 ts_tablet_manager.cc:1939] T a55352a03da243de9a8ff1f62a95cc35 P 143d3437e4be499fb78ad8ea64dfaaa4: Deleting consensus metadata
I20260925 02:45:39.557658 28396 ts_tablet_manager.cc:1905] T a55352a03da243de9a8ff1f62a95cc35 P 32c471b5298c474ebe1515814e5e6f35: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:45:39.558974 27879 catalog_manager.cc:4841] TS 143d3437e4be499fb78ad8ea64dfaaa4 (127.13.157.130:35399): tablet a55352a03da243de9a8ff1f62a95cc35 (table test-table [id=a34a880214154660a9c319ec9f3f53ca]) successfully deleted
W20260925 02:45:39.560029 27877 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.157.129:45983 (error 108)
I20260925 02:45:39.560322 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 28089
I20260925 02:45:39.560443 28396 ts_tablet_manager.cc:1918] T a55352a03da243de9a8ff1f62a95cc35 P 32c471b5298c474ebe1515814e5e6f35: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 02:45:39.560532 28396 log.cc:1196] T a55352a03da243de9a8ff1f62a95cc35 P 32c471b5298c474ebe1515814e5e6f35: Deleting WAL directory at /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-2/wal/wals/a55352a03da243de9a8ff1f62a95cc35
I20260925 02:45:39.560949 28396 ts_tablet_manager.cc:1939] T a55352a03da243de9a8ff1f62a95cc35 P 32c471b5298c474ebe1515814e5e6f35: Deleting consensus metadata
I20260925 02:45:39.562186 27878 catalog_manager.cc:4841] TS 32c471b5298c474ebe1515814e5e6f35 (127.13.157.131:40183): tablet a55352a03da243de9a8ff1f62a95cc35 (table test-table [id=a34a880214154660a9c319ec9f3f53ca]) successfully deleted
I20260925 02:45:39.563265 28369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:39.562967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.129:45983 (local address 127.0.0.1:60168)
0925 02:45:39.563088 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:39.563177 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.129:45983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260925 02:45:39.563421 27877 catalog_manager.cc:4584] TS 4ca40898617049bc9142388c6b5c8541 (127.13.157.129:45983): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a55352a03da243de9a8ff1f62a95cc35: Network error: Client connection negotiation failed: client connection to 127.13.157.129:45983: connect: Connection refused (error 111)
I20260925 02:45:39.566468 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 28225
I20260925 02:45:39.572314 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 27860
I20260925 02:45:39.578032 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.157.190:34431
--webserver_interface=127.13.157.190
--webserver_port=43715
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab
--principal=kudu/127.13.157.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:46509
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.157.190:34431
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:45:39.687626 28398 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:45:39.687908 28398 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:45:39.687965 28398 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:45:39.691938 28398 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:45:39.691995 28398 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:45:39.692021 28398 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:45:39.692042 28398 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:45:39.692065 28398 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:45:39.697250 28398 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:46509
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.13.157.190:34431
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab
--principal=kudu/127.13.157.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.157.190:34431
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.157.190
--webserver_port=43715
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28398
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:45:39.698323 28398 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:45:39.699376 28398 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:45:39.711613 28407 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:45:39.711615 28404 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:45:39.711740 28398 server_base.cc:1034] running on GCE node
W20260925 02:45:39.711617 28405 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:45:39.712447 28398 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:45:39.713606 28398 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:45:39.714763 28398 hybrid_clock.cc:648] HybridClock initialized: now 1790304339714740 us; error 35 us; skew 500 ppm
Sep 25 02:45:39 dist-test-slave-kznw krb5kdc[27847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304339, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:45:39.717670 28398 init.cc:377] Logged in from keytab as kudu/127.13.157.190@KRBTEST.COM (short username kudu)
I20260925 02:45:39.718622 28398 webserver.cc:460] Webserver started at http://127.13.157.190:43715/ using document root <none> and password file <none>
I20260925 02:45:39.719278 28398 fs_manager.cc:359] Metadata directory not provided
I20260925 02:45:39.719370 28398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:45:39.723268 28398 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260925 02:45:39.725401 28415 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:45:39.726649 28398 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260925 02:45:39.726754 28398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/master-0/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/master-0/wal
uuid: "dc06e289c66345e0b45649408d6da197"
format_stamp: "Formatted at 2026-09-25 02:45:38 on dist-test-slave-kznw"
server_key: "f97de62a81434f997a2859b500722b3b"
server_key_iv: "2e8faf4b230cedbf5bcde21f4850148c"
server_key_version: "encryptionkey@0"
I20260925 02:45:39.727411 28398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:45:39.741607 28398 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:45:39.744627 28398 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:45:39.744804 28398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:45:39.751763 28398 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.190:34431
I20260925 02:45:39.752043 28467 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.190:34431 every 8 connection(s)
I20260925 02:45:39.752871 28398 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
I20260925 02:45:39.754276 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 28398
I20260925 02:45:39.754747 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.157.129:45983
--local_ip_for_outbound_sockets=127.13.157.129
--tserver_master_addrs=127.13.157.190:34431
--webserver_port=40749
--webserver_interface=127.13.157.129
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.129.keytab
--principal=kudu/127.13.157.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:46509
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 02:45:39.758126 28468 sys_catalog.cc:263] Verifying existing consensus state
I20260925 02:45:39.759976 28468 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc06e289c66345e0b45649408d6da197: Bootstrap starting.
I20260925 02:45:39.777346 28468 log.cc:824] T 00000000000000000000000000000000 P dc06e289c66345e0b45649408d6da197: Log is configured to *not* fsync() on all Append() calls
I20260925 02:45:39.786367 28468 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc06e289c66345e0b45649408d6da197: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 02:45:39.786720 28468 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc06e289c66345e0b45649408d6da197: Bootstrap complete.
I20260925 02:45:39.790735 28468 raft_consensus.cc:348] T 00000000000000000000000000000000 P dc06e289c66345e0b45649408d6da197 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc06e289c66345e0b45649408d6da197" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 34431 } }
I20260925 02:45:39.791194 28468 raft_consensus.cc:729] T 00000000000000000000000000000000 P dc06e289c66345e0b45649408d6da197 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc06e289c66345e0b45649408d6da197, State: Initialized, Role: FOLLOWER
I20260925 02:45:39.791666 28468 consensus_queue.cc:260] T 00000000000000000000000000000000 P dc06e289c66345e0b45649408d6da197 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc06e289c66345e0b45649408d6da197" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 34431 } }
I20260925 02:45:39.791791 28468 raft_consensus.cc:388] T 00000000000000000000000000000000 P dc06e289c66345e0b45649408d6da197 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 02:45:39.791864 28468 raft_consensus.cc:482] T 00000000000000000000000000000000 P dc06e289c66345e0b45649408d6da197 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 02:45:39.791958 28468 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dc06e289c66345e0b45649408d6da197 [term 1 FOLLOWER]: Advancing to term 2
I20260925 02:45:39.793866 28468 raft_consensus.cc:504] T 00000000000000000000000000000000 P dc06e289c66345e0b45649408d6da197 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc06e289c66345e0b45649408d6da197" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 34431 } }
I20260925 02:45:39.794178 28468 leader_election.cc:302] T 00000000000000000000000000000000 P dc06e289c66345e0b45649408d6da197 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dc06e289c66345e0b45649408d6da197; no voters:
I20260925 02:45:39.794471 28468 leader_election.cc:288] T 00000000000000000000000000000000 P dc06e289c66345e0b45649408d6da197 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260925 02:45:39.794572 28472 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dc06e289c66345e0b45649408d6da197 [term 2 FOLLOWER]: Leader election won for term 2
I20260925 02:45:39.794747 28472 raft_consensus.cc:686] T 00000000000000000000000000000000 P dc06e289c66345e0b45649408d6da197 [term 2 LEADER]: Becoming Leader. State: Replica: dc06e289c66345e0b45649408d6da197, State: Running, Role: LEADER
I20260925 02:45:39.795063 28472 consensus_queue.cc:237] T 00000000000000000000000000000000 P dc06e289c66345e0b45649408d6da197 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc06e289c66345e0b45649408d6da197" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 34431 } }
I20260925 02:45:39.795769 28468 sys_catalog.cc:564] T 00000000000000000000000000000000 P dc06e289c66345e0b45649408d6da197 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:45:39.796504 28473 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc06e289c66345e0b45649408d6da197 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "dc06e289c66345e0b45649408d6da197" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc06e289c66345e0b45649408d6da197" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 34431 } } }
I20260925 02:45:39.796645 28473 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc06e289c66345e0b45649408d6da197 [sys.catalog]: This master's current role is: LEADER
I20260925 02:45:39.797168 28474 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc06e289c66345e0b45649408d6da197 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dc06e289c66345e0b45649408d6da197. Latest consensus state: current_term: 2 leader_uuid: "dc06e289c66345e0b45649408d6da197" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc06e289c66345e0b45649408d6da197" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 34431 } } }
I20260925 02:45:39.797323 28474 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc06e289c66345e0b45649408d6da197 [sys.catalog]: This master's current role is: LEADER
I20260925 02:45:39.797822 28481 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:45:39.802845 28481 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:45:39.803463 28481 catalog_manager.cc:1259] Loaded cluster ID: 303bcc0b7d1a473e887055648a7ee134
I20260925 02:45:39.803522 28481 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 02:45:39.804868 28481 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260925 02:45:39.809258 28481 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260925 02:45:39.814811 28481 catalog_manager.cc:1504] Loading token signing keys...
I20260925 02:45:39.815659 28481 catalog_manager.cc:5879] T 00000000000000000000000000000000 P dc06e289c66345e0b45649408d6da197: Loaded TSK: 0
I20260925 02:45:39.816929 28481 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260925 02:45:39.871495 28470 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:45:39.871755 28470 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:45:39.871798 28470 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:45:39.875897 28470 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:45:39.875962 28470 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:45:39.876020 28470 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.157.129
I20260925 02:45:39.881322 28470 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:46509
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.129.keytab
--principal=kudu/127.13.157.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.157.129:45983
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.157.129
--webserver_port=40749
--tserver_master_addrs=127.13.157.190:34431
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28470
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.157.129
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:45:39.882392 28470 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:45:39.883456 28470 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:45:39.894182 28495 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:45:39.894176 28496 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:45:39.894357 28470 server_base.cc:1034] running on GCE node
W20260925 02:45:39.894522 28498 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:45:39.895006 28470 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:45:39.895702 28470 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:45:39.896883 28470 hybrid_clock.cc:648] HybridClock initialized: now 1790304339896858 us; error 39 us; skew 500 ppm
Sep 25 02:45:39 dist-test-slave-kznw krb5kdc[27847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304339, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:45:39.899793 28470 init.cc:377] Logged in from keytab as kudu/127.13.157.129@KRBTEST.COM (short username kudu)
I20260925 02:45:39.900712 28470 webserver.cc:460] Webserver started at http://127.13.157.129:40749/ using document root <none> and password file <none>
I20260925 02:45:39.901253 28470 fs_manager.cc:359] Metadata directory not provided
I20260925 02:45:39.901336 28470 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:45:39.905337 28470 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260925 02:45:39.907644 28505 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:45:39.909091 28470 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260925 02:45:39.909202 28470 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-0/wal
uuid: "4ca40898617049bc9142388c6b5c8541"
format_stamp: "Formatted at 2026-09-25 02:45:38 on dist-test-slave-kznw"
server_key: "64b786325f8a3832ea4e8df017c64fcf"
server_key_iv: "d1c2e18f4d156da8822c91f076411eb6"
server_key_version: "encryptionkey@0"
I20260925 02:45:39.909778 28470 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:45:39.926456 28470 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:45:39.929807 28470 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:45:39.929988 28470 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:45:39.930660 28470 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:45:39.931977 28512 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260925 02:45:39.936303 28470 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260925 02:45:39.936374 28470 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20260925 02:45:39.936429 28470 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260925 02:45:39.938138 28470 ts_tablet_manager.cc:610] Registered 1 tablets
I20260925 02:45:39.938196 28470 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260925 02:45:39.938493 28512 tablet_bootstrap.cc:492] T a55352a03da243de9a8ff1f62a95cc35 P 4ca40898617049bc9142388c6b5c8541: Bootstrap starting.
I20260925 02:45:39.948616 28470 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.129:45983
I20260925 02:45:39.948660 28619 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.129:45983 every 8 connection(s)
I20260925 02:45:39.949901 28470 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-0/data/info.pb
I20260925 02:45:39.951579 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 28470
I20260925 02:45:39.952049 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.157.130:35399
--local_ip_for_outbound_sockets=127.13.157.130
--tserver_master_addrs=127.13.157.190:34431
--webserver_port=40957
--webserver_interface=127.13.157.130
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.130.keytab
--principal=kudu/127.13.157.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:46509
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 02:45:39.958616 28512 log.cc:824] T a55352a03da243de9a8ff1f62a95cc35 P 4ca40898617049bc9142388c6b5c8541: Log is configured to *not* fsync() on all Append() calls
Sep 25 02:45:39 dist-test-slave-kznw krb5kdc[27847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304339, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.129@KRBTEST.COM for kudu/127.13.157.190@KRBTEST.COM
I20260925 02:45:39.966375 28622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:39.952209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.129:38653 (local address 127.13.157.190:34431)
0925 02:45:39.952886 (+ 677us) server_negotiation.cc:206] Beginning negotiation
0925 02:45:39.952904 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0925 02:45:39.953414 (+ 510us) server_negotiation.cc:407] Connection header received
0925 02:45:39.954546 (+ 1132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:45:39.954557 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:45:39.954893 (+ 336us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:45:39.955252 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:45:39.956429 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:39.957400 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:39.958541 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:39.959186 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:39.959359 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:45:39.961697 (+ 2338us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:45:39.961702 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:45:39.961714 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:45:39.961744 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:45:39.963343 (+ 1599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:45:39.963832 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:45:39.963836 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:45:39.963840 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:45:39.963955 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:45:39.964283 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:45:39.964286 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:45:39.964288 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:45:39.964624 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:45:39.964737 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0925 02:45:39.965280 (+ 543us) server_negotiation.cc:299] Negotiation successful
0925 02:45:39.965534 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":532,"thread_start_us":108,"threads_started":1}
I20260925 02:45:39.966388 28623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:39.952164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:34431 (local address 127.13.157.129:38653)
0925 02:45:39.953218 (+ 1054us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:39.953275 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0925 02:45:39.954131 (+ 856us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:45:39.955329 (+ 1198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:45:39.955338 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:45:39.955749 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:45:39.956218 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:45:39.956265 (+ 47us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:45:39.957554 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:45:39.957556 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:45:39.958421 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:45:39.958431 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:45:39.959347 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:45:39.959352 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:45:39.959545 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:45:39.960099 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:45:39.960115 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:45:39.961508 (+ 1393us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:45:39.963453 (+ 1945us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:45:39.963458 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:45:39.963472 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:45:39.963716 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:45:39.964056 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:45:39.964059 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:45:39.964061 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:45:39.964192 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:45:39.964750 (+ 558us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:45:39.964757 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:45:39.964976 (+ 219us) client_negotiation.cc:769] Sending connection context
0925 02:45:39.965137 (+ 161us) client_negotiation.cc:240] Negotiation successful
0925 02:45:39.965468 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":816,"thread_start_us":163,"threads_started":1}
I20260925 02:45:39.967746 28512 tablet_bootstrap.cc:492] T a55352a03da243de9a8ff1f62a95cc35 P 4ca40898617049bc9142388c6b5c8541: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 02:45:39.968168 28512 tablet_bootstrap.cc:492] T a55352a03da243de9a8ff1f62a95cc35 P 4ca40898617049bc9142388c6b5c8541: Bootstrap complete.
I20260925 02:45:39.968799 28620 heartbeater.cc:347] Connected to a master server at 127.13.157.190:34431
I20260925 02:45:39.969017 28620 heartbeater.cc:464] Registering TS with master...
I20260925 02:45:39.969475 28620 heartbeater.cc:511] Master 127.13.157.190:34431 requested a full tablet report, sending...
I20260925 02:45:39.969520 28512 ts_tablet_manager.cc:1397] T a55352a03da243de9a8ff1f62a95cc35 P 4ca40898617049bc9142388c6b5c8541: Time spent bootstrapping tablet: real 0.031s user 0.021s sys 0.007s
I20260925 02:45:39.972251 28432 ts_manager.cc:194] Registered new tserver with Master: 4ca40898617049bc9142388c6b5c8541 (127.13.157.129:45983)
I20260925 02:45:39.972577 28512 raft_consensus.cc:348] T a55352a03da243de9a8ff1f62a95cc35 P 4ca40898617049bc9142388c6b5c8541 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "143d3437e4be499fb78ad8ea64dfaaa4" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 35399 } } peers { permanent_uuid: "4ca40898617049bc9142388c6b5c8541" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 45983 } } peers { permanent_uuid: "32c471b5298c474ebe1515814e5e6f35" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 40183 } }
I20260925 02:45:39.973114 28512 raft_consensus.cc:729] T a55352a03da243de9a8ff1f62a95cc35 P 4ca40898617049bc9142388c6b5c8541 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ca40898617049bc9142388c6b5c8541, State: Initialized, Role: FOLLOWER
I20260925 02:45:39.973608 28512 consensus_queue.cc:260] T a55352a03da243de9a8ff1f62a95cc35 P 4ca40898617049bc9142388c6b5c8541 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "143d3437e4be499fb78ad8ea64dfaaa4" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 35399 } } peers { permanent_uuid: "4ca40898617049bc9142388c6b5c8541" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 45983 } } peers { permanent_uuid: "32c471b5298c474ebe1515814e5e6f35" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 40183 } }
I20260925 02:45:39.974709 28512 ts_tablet_manager.cc:1428] T a55352a03da243de9a8ff1f62a95cc35 P 4ca40898617049bc9142388c6b5c8541: Time spent starting tablet: real 0.005s user 0.006s sys 0.001s
I20260925 02:45:39.976079 28432 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.157.129@KRBTEST.COM'} at 127.13.157.129:38653
I20260925 02:45:39.978564 28620 heartbeater.cc:503] Master 127.13.157.190:34431 was elected leader, sending a full tablet report...
Sep 25 02:45:39 dist-test-slave-kznw krb5kdc[27847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304339, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.190@KRBTEST.COM for kudu/127.13.157.129@KRBTEST.COM
I20260925 02:45:39.986438 28627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:39.975207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.129:45983 (local address 127.0.0.1:60184)
0925 02:45:39.975661 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:39.975714 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0925 02:45:39.975844 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:45:39.977396 (+ 1552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:45:39.977403 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:45:39.977839 (+ 436us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:45:39.977970 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:45:39.977979 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:45:39.979214 (+ 1235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:45:39.979218 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:45:39.980284 (+ 1066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:45:39.980296 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:45:39.980833 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:45:39.980837 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:45:39.980947 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:45:39.981454 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:45:39.981468 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:45:39.982881 (+ 1413us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:45:39.984782 (+ 1901us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:45:39.984790 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:45:39.984794 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:45:39.985005 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:45:39.985286 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:45:39.985288 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:45:39.985290 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:45:39.985362 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:45:39.985897 (+ 535us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:45:39.985903 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:45:39.986010 (+ 107us) client_negotiation.cc:769] Sending connection context
0925 02:45:39.986147 (+ 137us) client_negotiation.cc:240] Negotiation successful
0925 02:45:39.986300 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":124,"threads_started":1}
I20260925 02:45:39.986578 28628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:39.975317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60184 (local address 127.13.157.129:45983)
0925 02:45:39.976959 (+ 1642us) server_negotiation.cc:206] Beginning negotiation
0925 02:45:39.976968 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 02:45:39.976986 (+ 18us) server_negotiation.cc:407] Connection header received
0925 02:45:39.977038 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:45:39.977042 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:45:39.977216 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:45:39.977358 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:45:39.978110 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:39.979093 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:39.980408 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:39.980724 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:39.980807 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:45:39.983028 (+ 2221us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:45:39.983038 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:45:39.983051 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:45:39.983078 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:45:39.984679 (+ 1601us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:45:39.985101 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:45:39.985106 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:45:39.985108 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:45:39.985202 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:45:39.985430 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:45:39.985433 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:45:39.985435 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:45:39.985751 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:45:39.985885 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0925 02:45:39.986248 (+ 363us) server_negotiation.cc:299] Negotiation successful
0925 02:45:39.986419 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1549,"thread_start_us":88,"threads_started":1}
I20260925 02:45:39.987558 28554 tablet_service.cc:1514] Processing DeleteTablet for tablet a55352a03da243de9a8ff1f62a95cc35 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:45:39 UTC) from {username='kudu', principal='kudu/127.13.157.190@KRBTEST.COM'} at 127.0.0.1:60184
I20260925 02:45:39.988399 28629 tablet_replica.cc:330] stopping tablet replica
I20260925 02:45:39.988680 28629 raft_consensus.cc:2229] T a55352a03da243de9a8ff1f62a95cc35 P 4ca40898617049bc9142388c6b5c8541 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 02:45:39.988909 28629 raft_consensus.cc:2258] T a55352a03da243de9a8ff1f62a95cc35 P 4ca40898617049bc9142388c6b5c8541 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:45:39.989836 28629 ts_tablet_manager.cc:1905] T a55352a03da243de9a8ff1f62a95cc35 P 4ca40898617049bc9142388c6b5c8541: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:45:39.993564 28629 ts_tablet_manager.cc:1918] T a55352a03da243de9a8ff1f62a95cc35 P 4ca40898617049bc9142388c6b5c8541: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 02:45:39.993650 28629 log.cc:1196] T a55352a03da243de9a8ff1f62a95cc35 P 4ca40898617049bc9142388c6b5c8541: Deleting WAL directory at /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-0/wal/wals/a55352a03da243de9a8ff1f62a95cc35
I20260925 02:45:39.993899 28629 ts_tablet_manager.cc:1939] T a55352a03da243de9a8ff1f62a95cc35 P 4ca40898617049bc9142388c6b5c8541: Deleting consensus metadata
I20260925 02:45:39.995577 28417 catalog_manager.cc:4841] TS 4ca40898617049bc9142388c6b5c8541 (127.13.157.129:45983): tablet a55352a03da243de9a8ff1f62a95cc35 (table test-table [id=a34a880214154660a9c319ec9f3f53ca]) successfully deleted
W20260925 02:45:40.079991 28624 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:45:40.080303 28624 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:45:40.080368 28624 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:45:40.084529 28624 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:45:40.084614 28624 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:45:40.084697 28624 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.157.130
I20260925 02:45:40.090261 28624 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:46509
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.130.keytab
--principal=kudu/127.13.157.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.157.130:35399
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.157.130
--webserver_port=40957
--tserver_master_addrs=127.13.157.190:34431
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28624
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.157.130
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:45:40.091517 28624 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:45:40.092574 28624 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:45:40.102530 28637 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:45:40.102523 28634 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:45:40.102788 28635 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:45:40.102895 28624 server_base.cc:1034] running on GCE node
I20260925 02:45:40.103343 28624 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:45:40.104044 28624 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:45:40.105235 28624 hybrid_clock.cc:648] HybridClock initialized: now 1790304340105200 us; error 43 us; skew 500 ppm
Sep 25 02:45:40 dist-test-slave-kznw krb5kdc[27847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304340, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:45:40.108043 28624 init.cc:377] Logged in from keytab as kudu/127.13.157.130@KRBTEST.COM (short username kudu)
I20260925 02:45:40.108989 28624 webserver.cc:460] Webserver started at http://127.13.157.130:40957/ using document root <none> and password file <none>
I20260925 02:45:40.109544 28624 fs_manager.cc:359] Metadata directory not provided
I20260925 02:45:40.109624 28624 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:45:40.113792 28624 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260925 02:45:40.116119 28644 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:45:40.117486 28624 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 02:45:40.117633 28624 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-1/wal
uuid: "143d3437e4be499fb78ad8ea64dfaaa4"
format_stamp: "Formatted at 2026-09-25 02:45:39 on dist-test-slave-kznw"
server_key: "9b52e7135c42992013322d738375c784"
server_key_iv: "c18cb6d8e6c95eb58062bb8b1be47467"
server_key_version: "encryptionkey@0"
I20260925 02:45:40.118209 28624 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:45:40.142452 28624 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:45:40.145534 28624 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:45:40.145730 28624 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:45:40.146344 28624 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:45:40.147253 28624 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:45:40.147301 28624 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:45:40.147356 28624 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:45:40.147385 28624 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:45:40.155756 28624 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.130:35399
I20260925 02:45:40.155825 28757 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.130:35399 every 8 connection(s)
I20260925 02:45:40.156769 28624 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-1/data/info.pb
Sep 25 02:45:40 dist-test-slave-kznw krb5kdc[27847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304340, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.130@KRBTEST.COM for kudu/127.13.157.190@KRBTEST.COM
I20260925 02:45:40.167070 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 28624
I20260925 02:45:40.167549 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.157.131:40183
--local_ip_for_outbound_sockets=127.13.157.131
--tserver_master_addrs=127.13.157.190:34431
--webserver_port=35755
--webserver_interface=127.13.157.131
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.131.keytab
--principal=kudu/127.13.157.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:46509
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 02:45:40.170248 28622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:40.158600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.130:50999 (local address 127.13.157.190:34431)
0925 02:45:40.158830 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0925 02:45:40.158834 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:45:40.159634 (+ 800us) server_negotiation.cc:407] Connection header received
0925 02:45:40.160964 (+ 1330us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:45:40.160967 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:45:40.161027 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:45:40.161101 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:45:40.162274 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:40.162795 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:40.163943 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:40.164171 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:40.164269 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:45:40.166548 (+ 2279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:45:40.166551 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:45:40.166553 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:45:40.166577 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:45:40.168116 (+ 1539us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:45:40.168587 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:45:40.168590 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:45:40.168592 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:45:40.168680 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:45:40.169135 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:45:40.169137 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:45:40.169138 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:45:40.169312 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:45:40.169387 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0925 02:45:40.170016 (+ 629us) server_negotiation.cc:299] Negotiation successful
0925 02:45:40.170123 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133}
I20260925 02:45:40.171458 28760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:40.159026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:34431 (local address 127.13.157.130:50999)
0925 02:45:40.159478 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:40.159518 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0925 02:45:40.160757 (+ 1239us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:45:40.161287 (+ 530us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:45:40.161294 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:45:40.161653 (+ 359us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:45:40.162118 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:45:40.162131 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:45:40.162935 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:45:40.162939 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:45:40.163819 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:45:40.163828 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:45:40.164315 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:45:40.164319 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:45:40.164479 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:45:40.165015 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:45:40.165031 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:45:40.166395 (+ 1364us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:45:40.168227 (+ 1832us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:45:40.168232 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:45:40.168244 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:45:40.168497 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:45:40.168813 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:45:40.168817 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:45:40.168820 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:45:40.169031 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:45:40.169407 (+ 376us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:45:40.169414 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:45:40.169759 (+ 345us) client_negotiation.cc:769] Sending connection context
0925 02:45:40.169992 (+ 233us) client_negotiation.cc:240] Negotiation successful
0925 02:45:40.170334 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":99,"threads_started":1}
I20260925 02:45:40.172878 28758 heartbeater.cc:347] Connected to a master server at 127.13.157.190:34431
I20260925 02:45:40.173128 28758 heartbeater.cc:464] Registering TS with master...
I20260925 02:45:40.173622 28758 heartbeater.cc:511] Master 127.13.157.190:34431 requested a full tablet report, sending...
I20260925 02:45:40.174813 28432 ts_manager.cc:194] Registered new tserver with Master: 143d3437e4be499fb78ad8ea64dfaaa4 (127.13.157.130:35399)
I20260925 02:45:40.175385 28432 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.157.130@KRBTEST.COM'} at 127.13.157.130:50999
W20260925 02:45:40.282017 28761 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:45:40.282277 28761 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:45:40.282320 28761 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:45:40.286852 28761 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:45:40.286948 28761 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:45:40.287036 28761 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.157.131
I20260925 02:45:40.292624 28761 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:46509
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.131.keytab
--principal=kudu/127.13.157.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.157.131:40183
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.157.131
--webserver_port=35755
--tserver_master_addrs=127.13.157.190:34431
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28761
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.157.131
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:45:40.293653 28761 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:45:40.294679 28761 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:45:40.304544 28769 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:45:40.304557 28766 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:45:40.304734 28761 server_base.cc:1034] running on GCE node
W20260925 02:45:40.304560 28767 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:45:40.305259 28761 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:45:40.305927 28761 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:45:40.307081 28761 hybrid_clock.cc:648] HybridClock initialized: now 1790304340307067 us; error 44 us; skew 500 ppm
Sep 25 02:45:40 dist-test-slave-kznw krb5kdc[27847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304340, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:45:40.310029 28761 init.cc:377] Logged in from keytab as kudu/127.13.157.131@KRBTEST.COM (short username kudu)
I20260925 02:45:40.311041 28761 webserver.cc:460] Webserver started at http://127.13.157.131:35755/ using document root <none> and password file <none>
I20260925 02:45:40.311584 28761 fs_manager.cc:359] Metadata directory not provided
I20260925 02:45:40.311669 28761 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:45:40.315912 28761 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.003s
I20260925 02:45:40.318164 28776 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:45:40.319556 28761 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 02:45:40.319658 28761 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-2/wal
uuid: "32c471b5298c474ebe1515814e5e6f35"
format_stamp: "Formatted at 2026-09-25 02:45:39 on dist-test-slave-kznw"
server_key: "97d2bc30a81a3deef3cda089102dc495"
server_key_iv: "e5cedfb7525abc455d8d832ac3cd7571"
server_key_version: "encryptionkey@0"
I20260925 02:45:40.320209 28761 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:45:40.339437 28761 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:45:40.342711 28761 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:45:40.342881 28761 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:45:40.343470 28761 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:45:40.344285 28761 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:45:40.344328 28761 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:45:40.344367 28761 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:45:40.344389 28761 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:45:40.353444 28761 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.131:40183
I20260925 02:45:40.353487 28889 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.131:40183 every 8 connection(s)
I20260925 02:45:40.354540 28761 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-2/data/info.pb
I20260925 02:45:40.354944 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 28761
I20260925 02:45:40.364374 28622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:40.355723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41910 (local address 127.13.157.190:34431)
0925 02:45:40.355869 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0925 02:45:40.355875 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 02:45:40.356024 (+ 149us) server_negotiation.cc:407] Connection header received
0925 02:45:40.356158 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:45:40.356161 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:45:40.356215 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:45:40.356310 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:45:40.357137 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:40.357887 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:40.359175 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:40.359412 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:40.359541 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:45:40.361144 (+ 1603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:45:40.361149 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:45:40.361152 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:45:40.361181 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:45:40.362553 (+ 1372us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:45:40.363011 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:45:40.363014 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:45:40.363016 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:45:40.363119 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:45:40.363374 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:45:40.363378 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:45:40.363379 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:45:40.363548 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:45:40.363748 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0925 02:45:40.364077 (+ 329us) server_negotiation.cc:299] Negotiation successful
0925 02:45:40.364215 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
Sep 25 02:45:40 dist-test-slave-kznw krb5kdc[27847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304340, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.131@KRBTEST.COM for kudu/127.13.157.190@KRBTEST.COM
I20260925 02:45:40.368666 28893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:40.356480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.131:44715 (local address 127.13.157.190:34431)
0925 02:45:40.356708 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0925 02:45:40.356713 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:45:40.357597 (+ 884us) server_negotiation.cc:407] Connection header received
0925 02:45:40.359268 (+ 1671us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:45:40.359271 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:45:40.359336 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:45:40.359415 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:45:40.360629 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:40.361144 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:40.362556 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:40.362795 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:40.362880 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:45:40.365332 (+ 2452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:45:40.365336 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:45:40.365338 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:45:40.365363 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:45:40.366793 (+ 1430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:45:40.367308 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:45:40.367311 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:45:40.367312 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:45:40.367394 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:45:40.367685 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:45:40.367687 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:45:40.367688 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:45:40.367857 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:45:40.367929 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0925 02:45:40.368424 (+ 495us) server_negotiation.cc:299] Negotiation successful
0925 02:45:40.368527 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":67,"threads_started":1}
I20260925 02:45:40.369455 28894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:40.356961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:34431 (local address 127.13.157.131:44715)
0925 02:45:40.357461 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:40.357505 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0925 02:45:40.359052 (+ 1547us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:45:40.359599 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:45:40.359606 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:45:40.359974 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:45:40.360492 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:45:40.360505 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:45:40.361261 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:45:40.361264 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:45:40.362411 (+ 1147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:45:40.362422 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:45:40.362971 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:45:40.362976 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:45:40.363140 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:45:40.363623 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:45:40.363640 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:45:40.365167 (+ 1527us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:45:40.366893 (+ 1726us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:45:40.366898 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:45:40.366938 (+ 40us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:45:40.367200 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:45:40.367473 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:45:40.367475 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:45:40.367477 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:45:40.367610 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:45:40.367932 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:45:40.367937 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:45:40.368168 (+ 231us) client_negotiation.cc:769] Sending connection context
0925 02:45:40.368347 (+ 179us) client_negotiation.cc:240] Negotiation successful
0925 02:45:40.368607 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":139,"threads_started":1}
I20260925 02:45:40.370509 28890 heartbeater.cc:347] Connected to a master server at 127.13.157.190:34431
I20260925 02:45:40.370756 28890 heartbeater.cc:464] Registering TS with master...
I20260925 02:45:40.371304 28890 heartbeater.cc:511] Master 127.13.157.190:34431 requested a full tablet report, sending...
I20260925 02:45:40.372268 28432 ts_manager.cc:194] Registered new tserver with Master: 32c471b5298c474ebe1515814e5e6f35 (127.13.157.131:40183)
I20260925 02:45:40.372834 28432 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.157.131@KRBTEST.COM'} at 127.13.157.131:44715
I20260925 02:45:40.378077 13942 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 02:45:40.382259 28893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:40.378662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41922 (local address 127.13.157.190:34431)
0925 02:45:40.378806 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0925 02:45:40.378811 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:45:40.379047 (+ 236us) server_negotiation.cc:407] Connection header received
0925 02:45:40.379183 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:45:40.379185 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:45:40.379289 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:45:40.379388 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 02:45:40.379571 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:40.380085 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:40.380922 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:40.381152 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:40.381247 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:45:40.381531 (+ 284us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 02:45:40.381916 (+ 385us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 02:45:40.382001 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0925 02:45:40.382048 (+ 47us) server_negotiation.cc:299] Negotiation successful
0925 02:45:40.382104 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260925 02:45:40.382889 28432 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:41922:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 02:45:40.386189 28432 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 02:45:40.394953 28554 tablet_service.cc:1467] Processing CreateTablet for tablet 2582c7b01cb645858bebdc8df683334a (DEFAULT_TABLE table=test-table [id=2831658670ec445c8ea7c21d05e843ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:45:40.395919 28554 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2582c7b01cb645858bebdc8df683334a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:45:40.399142 28512 tablet_bootstrap.cc:492] T 2582c7b01cb645858bebdc8df683334a P 4ca40898617049bc9142388c6b5c8541: Bootstrap starting.
I20260925 02:45:40.399572 28899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:40.394033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.130:35399 (local address 127.0.0.1:37388)
0925 02:45:40.394366 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:40.394378 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 02:45:40.394463 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:45:40.395335 (+ 872us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:45:40.395338 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:45:40.395356 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:45:40.395454 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:45:40.395461 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:45:40.397147 (+ 1686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:45:40.397151 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:45:40.398250 (+ 1099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:45:40.398258 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:45:40.399106 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:45:40.399110 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:45:40.399212 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:45:40.399269 (+ 57us) client_negotiation.cc:769] Sending connection context
0925 02:45:40.399340 (+ 71us) client_negotiation.cc:240] Negotiation successful
0925 02:45:40.399422 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":125,"threads_started":1}
I20260925 02:45:40.400143 28512 tablet_bootstrap.cc:654] T 2582c7b01cb645858bebdc8df683334a P 4ca40898617049bc9142388c6b5c8541: Neither blocks nor log segments found. Creating new log.
I20260925 02:45:40.400243 28901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:40.394362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37388 (local address 127.13.157.130:35399)
0925 02:45:40.394746 (+ 384us) server_negotiation.cc:206] Beginning negotiation
0925 02:45:40.394758 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 02:45:40.394781 (+ 23us) server_negotiation.cc:407] Connection header received
0925 02:45:40.394852 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:45:40.394857 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:45:40.395087 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:45:40.395605 (+ 518us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:45:40.395903 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:40.396998 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:40.398364 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:40.398966 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:40.399792 (+ 826us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:45:40.399928 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0925 02:45:40.399984 (+ 56us) server_negotiation.cc:299] Negotiation successful
0925 02:45:40.400084 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":108,"threads_started":1}
I20260925 02:45:40.400367 28627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:40.393770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.131:40183 (local address 127.0.0.1:51444)
0925 02:45:40.393931 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:40.393959 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0925 02:45:40.394048 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:45:40.395091 (+ 1043us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:45:40.395094 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:45:40.395113 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:45:40.395232 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:45:40.395240 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:45:40.396684 (+ 1444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:45:40.396688 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:45:40.397938 (+ 1250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:45:40.397951 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:45:40.399945 (+ 1994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:45:40.399949 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:45:40.400049 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:45:40.400072 (+ 23us) client_negotiation.cc:769] Sending connection context
0925 02:45:40.400145 (+ 73us) client_negotiation.cc:240] Negotiation successful
0925 02:45:40.400245 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":78}
I20260925 02:45:40.401108 28900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:40.394051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51444 (local address 127.13.157.131:40183)
0925 02:45:40.394615 (+ 564us) server_negotiation.cc:206] Beginning negotiation
0925 02:45:40.394627 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 02:45:40.394656 (+ 29us) server_negotiation.cc:407] Connection header received
0925 02:45:40.394733 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:45:40.394737 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:45:40.394951 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:45:40.395103 (+ 152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:45:40.395363 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:40.396558 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:40.398055 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:40.398938 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:40.399063 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:45:40.399202 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0925 02:45:40.400869 (+ 1667us) server_negotiation.cc:299] Negotiation successful
0925 02:45:40.400970 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":454,"thread_start_us":201,"threads_started":1}
I20260925 02:45:40.402127 28512 tablet_bootstrap.cc:492] T 2582c7b01cb645858bebdc8df683334a P 4ca40898617049bc9142388c6b5c8541: No bootstrap required, opened a new log
I20260925 02:45:40.402230 28512 ts_tablet_manager.cc:1397] T 2582c7b01cb645858bebdc8df683334a P 4ca40898617049bc9142388c6b5c8541: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260925 02:45:40.402762 28512 raft_consensus.cc:348] T 2582c7b01cb645858bebdc8df683334a P 4ca40898617049bc9142388c6b5c8541 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c471b5298c474ebe1515814e5e6f35" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 40183 } } peers { permanent_uuid: "143d3437e4be499fb78ad8ea64dfaaa4" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 35399 } } peers { permanent_uuid: "4ca40898617049bc9142388c6b5c8541" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 45983 } }
I20260925 02:45:40.402896 28512 raft_consensus.cc:374] T 2582c7b01cb645858bebdc8df683334a P 4ca40898617049bc9142388c6b5c8541 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:45:40.402980 28512 raft_consensus.cc:729] T 2582c7b01cb645858bebdc8df683334a P 4ca40898617049bc9142388c6b5c8541 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ca40898617049bc9142388c6b5c8541, State: Initialized, Role: FOLLOWER
I20260925 02:45:40.403142 28512 consensus_queue.cc:260] T 2582c7b01cb645858bebdc8df683334a P 4ca40898617049bc9142388c6b5c8541 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c471b5298c474ebe1515814e5e6f35" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 40183 } } peers { permanent_uuid: "143d3437e4be499fb78ad8ea64dfaaa4" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 35399 } } peers { permanent_uuid: "4ca40898617049bc9142388c6b5c8541" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 45983 } }
I20260925 02:45:40.403568 28512 ts_tablet_manager.cc:1428] T 2582c7b01cb645858bebdc8df683334a P 4ca40898617049bc9142388c6b5c8541: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20260925 02:45:40.403959 28692 tablet_service.cc:1467] Processing CreateTablet for tablet 2582c7b01cb645858bebdc8df683334a (DEFAULT_TABLE table=test-table [id=2831658670ec445c8ea7c21d05e843ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:45:40.403951 28824 tablet_service.cc:1467] Processing CreateTablet for tablet 2582c7b01cb645858bebdc8df683334a (DEFAULT_TABLE table=test-table [id=2831658670ec445c8ea7c21d05e843ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:45:40.404929 28692 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2582c7b01cb645858bebdc8df683334a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:45:40.404989 28824 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2582c7b01cb645858bebdc8df683334a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:45:40.411485 28903 tablet_bootstrap.cc:492] T 2582c7b01cb645858bebdc8df683334a P 32c471b5298c474ebe1515814e5e6f35: Bootstrap starting.
I20260925 02:45:40.412469 28904 tablet_bootstrap.cc:492] T 2582c7b01cb645858bebdc8df683334a P 143d3437e4be499fb78ad8ea64dfaaa4: Bootstrap starting.
I20260925 02:45:40.414000 28903 tablet_bootstrap.cc:654] T 2582c7b01cb645858bebdc8df683334a P 32c471b5298c474ebe1515814e5e6f35: Neither blocks nor log segments found. Creating new log.
I20260925 02:45:40.414718 28903 log.cc:824] T 2582c7b01cb645858bebdc8df683334a P 32c471b5298c474ebe1515814e5e6f35: Log is configured to *not* fsync() on all Append() calls
I20260925 02:45:40.414796 28904 tablet_bootstrap.cc:654] T 2582c7b01cb645858bebdc8df683334a P 143d3437e4be499fb78ad8ea64dfaaa4: Neither blocks nor log segments found. Creating new log.
I20260925 02:45:40.415578 28904 log.cc:824] T 2582c7b01cb645858bebdc8df683334a P 143d3437e4be499fb78ad8ea64dfaaa4: Log is configured to *not* fsync() on all Append() calls
I20260925 02:45:40.416736 28903 tablet_bootstrap.cc:492] T 2582c7b01cb645858bebdc8df683334a P 32c471b5298c474ebe1515814e5e6f35: No bootstrap required, opened a new log
I20260925 02:45:40.416929 28903 ts_tablet_manager.cc:1397] T 2582c7b01cb645858bebdc8df683334a P 32c471b5298c474ebe1515814e5e6f35: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260925 02:45:40.417357 28904 tablet_bootstrap.cc:492] T 2582c7b01cb645858bebdc8df683334a P 143d3437e4be499fb78ad8ea64dfaaa4: No bootstrap required, opened a new log
I20260925 02:45:40.417522 28904 ts_tablet_manager.cc:1397] T 2582c7b01cb645858bebdc8df683334a P 143d3437e4be499fb78ad8ea64dfaaa4: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260925 02:45:40.420116 28903 raft_consensus.cc:348] T 2582c7b01cb645858bebdc8df683334a P 32c471b5298c474ebe1515814e5e6f35 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c471b5298c474ebe1515814e5e6f35" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 40183 } } peers { permanent_uuid: "143d3437e4be499fb78ad8ea64dfaaa4" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 35399 } } peers { permanent_uuid: "4ca40898617049bc9142388c6b5c8541" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 45983 } }
I20260925 02:45:40.420398 28903 raft_consensus.cc:374] T 2582c7b01cb645858bebdc8df683334a P 32c471b5298c474ebe1515814e5e6f35 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:45:40.420460 28903 raft_consensus.cc:729] T 2582c7b01cb645858bebdc8df683334a P 32c471b5298c474ebe1515814e5e6f35 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32c471b5298c474ebe1515814e5e6f35, State: Initialized, Role: FOLLOWER
I20260925 02:45:40.420723 28904 raft_consensus.cc:348] T 2582c7b01cb645858bebdc8df683334a P 143d3437e4be499fb78ad8ea64dfaaa4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c471b5298c474ebe1515814e5e6f35" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 40183 } } peers { permanent_uuid: "143d3437e4be499fb78ad8ea64dfaaa4" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 35399 } } peers { permanent_uuid: "4ca40898617049bc9142388c6b5c8541" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 45983 } }
I20260925 02:45:40.420979 28904 raft_consensus.cc:374] T 2582c7b01cb645858bebdc8df683334a P 143d3437e4be499fb78ad8ea64dfaaa4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:45:40.420946 28903 consensus_queue.cc:260] T 2582c7b01cb645858bebdc8df683334a P 32c471b5298c474ebe1515814e5e6f35 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c471b5298c474ebe1515814e5e6f35" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 40183 } } peers { permanent_uuid: "143d3437e4be499fb78ad8ea64dfaaa4" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 35399 } } peers { permanent_uuid: "4ca40898617049bc9142388c6b5c8541" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 45983 } }
I20260925 02:45:40.421039 28904 raft_consensus.cc:729] T 2582c7b01cb645858bebdc8df683334a P 143d3437e4be499fb78ad8ea64dfaaa4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 143d3437e4be499fb78ad8ea64dfaaa4, State: Initialized, Role: FOLLOWER
I20260925 02:45:40.421427 28904 consensus_queue.cc:260] T 2582c7b01cb645858bebdc8df683334a P 143d3437e4be499fb78ad8ea64dfaaa4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c471b5298c474ebe1515814e5e6f35" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 40183 } } peers { permanent_uuid: "143d3437e4be499fb78ad8ea64dfaaa4" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 35399 } } peers { permanent_uuid: "4ca40898617049bc9142388c6b5c8541" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 45983 } }
I20260925 02:45:40.421653 28890 heartbeater.cc:503] Master 127.13.157.190:34431 was elected leader, sending a full tablet report...
I20260925 02:45:40.422032 28903 ts_tablet_manager.cc:1428] T 2582c7b01cb645858bebdc8df683334a P 32c471b5298c474ebe1515814e5e6f35: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260925 02:45:40.422094 28758 heartbeater.cc:503] Master 127.13.157.190:34431 was elected leader, sending a full tablet report...
I20260925 02:45:40.422403 28904 ts_tablet_manager.cc:1428] T 2582c7b01cb645858bebdc8df683334a P 143d3437e4be499fb78ad8ea64dfaaa4: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
W20260925 02:45:40.451293 28621 tablet.cc:2369] T 2582c7b01cb645858bebdc8df683334a P 4ca40898617049bc9142388c6b5c8541: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 02:45:40.471788 28626 raft_consensus.cc:482] T 2582c7b01cb645858bebdc8df683334a P 4ca40898617049bc9142388c6b5c8541 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 02:45:40.472007 28626 raft_consensus.cc:504] T 2582c7b01cb645858bebdc8df683334a P 4ca40898617049bc9142388c6b5c8541 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c471b5298c474ebe1515814e5e6f35" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 40183 } } peers { permanent_uuid: "143d3437e4be499fb78ad8ea64dfaaa4" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 35399 } } peers { permanent_uuid: "4ca40898617049bc9142388c6b5c8541" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 45983 } }
I20260925 02:45:40.473184 28626 leader_election.cc:288] T 2582c7b01cb645858bebdc8df683334a P 4ca40898617049bc9142388c6b5c8541 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 32c471b5298c474ebe1515814e5e6f35 (127.13.157.131:40183), 143d3437e4be499fb78ad8ea64dfaaa4 (127.13.157.130:35399)
I20260925 02:45:40.478546 28900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:40.473520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.129:34771 (local address 127.13.157.131:40183)
0925 02:45:40.473654 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0925 02:45:40.473658 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:45:40.473993 (+ 335us) server_negotiation.cc:407] Connection header received
0925 02:45:40.474093 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:45:40.474096 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:45:40.474147 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:45:40.474238 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:45:40.474593 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:40.475418 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:40.477164 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:40.477790 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:40.477921 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:45:40.477976 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0925 02:45:40.478330 (+ 354us) server_negotiation.cc:299] Negotiation successful
0925 02:45:40.478413 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260925 02:45:40.478543 28901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:40.473556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.129:45887 (local address 127.13.157.130:35399)
0925 02:45:40.473693 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0925 02:45:40.473697 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:45:40.473893 (+ 196us) server_negotiation.cc:407] Connection header received
0925 02:45:40.474093 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:45:40.474096 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:45:40.474151 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:45:40.474240 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:45:40.474593 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:40.475418 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:40.477165 (+ 1747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:40.477829 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:40.477927 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:45:40.477975 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0925 02:45:40.478330 (+ 355us) server_negotiation.cc:299] Negotiation successful
0925 02:45:40.478413 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260925 02:45:40.478580 28909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:40.473400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.130:35399 (local address 127.13.157.129:45887)
0925 02:45:40.473804 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:40.473819 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 02:45:40.473958 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:45:40.474255 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:45:40.474258 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:45:40.474294 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:45:40.474488 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:45:40.474496 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:45:40.475531 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:45:40.475535 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:45:40.477029 (+ 1494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:45:40.477041 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:45:40.477938 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:45:40.477942 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:45:40.478137 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:45:40.478199 (+ 62us) client_negotiation.cc:769] Sending connection context
0925 02:45:40.478282 (+ 83us) client_negotiation.cc:240] Negotiation successful
0925 02:45:40.478380 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":338,"mutex_wait_us":240,"spinlock_wait_cycles":6400,"thread_start_us":82,"threads_started":1}
I20260925 02:45:40.478583 28910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:40.473400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.131:40183 (local address 127.13.157.129:34771)
0925 02:45:40.473903 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:40.473914 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 02:45:40.474000 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:45:40.474255 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:45:40.474258 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:45:40.474291 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:45:40.474490 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:45:40.474496 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:45:40.475536 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:45:40.475539 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:45:40.477029 (+ 1490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:45:40.477041 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:45:40.477938 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:45:40.477943 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:45:40.478121 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:45:40.478198 (+ 77us) client_negotiation.cc:769] Sending connection context
0925 02:45:40.478282 (+ 84us) client_negotiation.cc:240] Negotiation successful
0925 02:45:40.478380 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":449,"mutex_wait_us":233,"spinlock_wait_cycles":8832,"thread_start_us":93,"threads_started":1}
I20260925 02:45:40.479305 28712 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2582c7b01cb645858bebdc8df683334a" candidate_uuid: "4ca40898617049bc9142388c6b5c8541" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "143d3437e4be499fb78ad8ea64dfaaa4" is_pre_election: true
I20260925 02:45:40.479492 28844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2582c7b01cb645858bebdc8df683334a" candidate_uuid: "4ca40898617049bc9142388c6b5c8541" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32c471b5298c474ebe1515814e5e6f35" is_pre_election: true
I20260925 02:45:40.479641 28712 raft_consensus.cc:2454] T 2582c7b01cb645858bebdc8df683334a P 143d3437e4be499fb78ad8ea64dfaaa4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4ca40898617049bc9142388c6b5c8541 in term 0.
I20260925 02:45:40.479822 28844 raft_consensus.cc:2454] T 2582c7b01cb645858bebdc8df683334a P 32c471b5298c474ebe1515814e5e6f35 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4ca40898617049bc9142388c6b5c8541 in term 0.
I20260925 02:45:40.480127 28509 leader_election.cc:302] T 2582c7b01cb645858bebdc8df683334a P 4ca40898617049bc9142388c6b5c8541 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 143d3437e4be499fb78ad8ea64dfaaa4, 4ca40898617049bc9142388c6b5c8541; no voters:
I20260925 02:45:40.480453 28626 raft_consensus.cc:2781] T 2582c7b01cb645858bebdc8df683334a P 4ca40898617049bc9142388c6b5c8541 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 02:45:40.480546 28626 raft_consensus.cc:482] T 2582c7b01cb645858bebdc8df683334a P 4ca40898617049bc9142388c6b5c8541 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 02:45:40.480608 28626 raft_consensus.cc:3037] T 2582c7b01cb645858bebdc8df683334a P 4ca40898617049bc9142388c6b5c8541 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:45:40.481698 28626 raft_consensus.cc:504] T 2582c7b01cb645858bebdc8df683334a P 4ca40898617049bc9142388c6b5c8541 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c471b5298c474ebe1515814e5e6f35" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 40183 } } peers { permanent_uuid: "143d3437e4be499fb78ad8ea64dfaaa4" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 35399 } } peers { permanent_uuid: "4ca40898617049bc9142388c6b5c8541" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 45983 } }
I20260925 02:45:40.482040 28626 leader_election.cc:288] T 2582c7b01cb645858bebdc8df683334a P 4ca40898617049bc9142388c6b5c8541 [CANDIDATE]: Term 1 election: Requested vote from peers 32c471b5298c474ebe1515814e5e6f35 (127.13.157.131:40183), 143d3437e4be499fb78ad8ea64dfaaa4 (127.13.157.130:35399)
I20260925 02:45:40.482376 28712 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2582c7b01cb645858bebdc8df683334a" candidate_uuid: "4ca40898617049bc9142388c6b5c8541" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "143d3437e4be499fb78ad8ea64dfaaa4"
I20260925 02:45:40.482415 28844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2582c7b01cb645858bebdc8df683334a" candidate_uuid: "4ca40898617049bc9142388c6b5c8541" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32c471b5298c474ebe1515814e5e6f35"
I20260925 02:45:40.482501 28712 raft_consensus.cc:3037] T 2582c7b01cb645858bebdc8df683334a P 143d3437e4be499fb78ad8ea64dfaaa4 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:45:40.482570 28844 raft_consensus.cc:3037] T 2582c7b01cb645858bebdc8df683334a P 32c471b5298c474ebe1515814e5e6f35 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:45:40.483667 28712 raft_consensus.cc:2454] T 2582c7b01cb645858bebdc8df683334a P 143d3437e4be499fb78ad8ea64dfaaa4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ca40898617049bc9142388c6b5c8541 in term 1.
I20260925 02:45:40.483680 28844 raft_consensus.cc:2454] T 2582c7b01cb645858bebdc8df683334a P 32c471b5298c474ebe1515814e5e6f35 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ca40898617049bc9142388c6b5c8541 in term 1.
I20260925 02:45:40.483949 28508 leader_election.cc:302] T 2582c7b01cb645858bebdc8df683334a P 4ca40898617049bc9142388c6b5c8541 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 143d3437e4be499fb78ad8ea64dfaaa4, 32c471b5298c474ebe1515814e5e6f35, 4ca40898617049bc9142388c6b5c8541; no voters:
I20260925 02:45:40.484196 28626 raft_consensus.cc:2781] T 2582c7b01cb645858bebdc8df683334a P 4ca40898617049bc9142388c6b5c8541 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:45:40.484431 28626 raft_consensus.cc:686] T 2582c7b01cb645858bebdc8df683334a P 4ca40898617049bc9142388c6b5c8541 [term 1 LEADER]: Becoming Leader. State: Replica: 4ca40898617049bc9142388c6b5c8541, State: Running, Role: LEADER
I20260925 02:45:40.484709 28626 consensus_queue.cc:237] T 2582c7b01cb645858bebdc8df683334a P 4ca40898617049bc9142388c6b5c8541 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c471b5298c474ebe1515814e5e6f35" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 40183 } } peers { permanent_uuid: "143d3437e4be499fb78ad8ea64dfaaa4" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 35399 } } peers { permanent_uuid: "4ca40898617049bc9142388c6b5c8541" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 45983 } }
I20260925 02:45:40.487884 28431 catalog_manager.cc:5495] T 2582c7b01cb645858bebdc8df683334a P 4ca40898617049bc9142388c6b5c8541 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4ca40898617049bc9142388c6b5c8541 (127.13.157.129). New cstate: current_term: 1 leader_uuid: "4ca40898617049bc9142388c6b5c8541" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c471b5298c474ebe1515814e5e6f35" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 40183 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "143d3437e4be499fb78ad8ea64dfaaa4" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 35399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4ca40898617049bc9142388c6b5c8541" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 45983 } health_report { overall_health: HEALTHY } } }
I20260925 02:45:40.510901 28914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:40.507422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60198 (local address 127.13.157.129:45983)
0925 02:45:40.507699 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0925 02:45:40.507703 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:45:40.507715 (+ 12us) server_negotiation.cc:407] Connection header received
0925 02:45:40.507786 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:45:40.507788 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:45:40.507844 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:45:40.507943 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 02:45:40.508180 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:40.508695 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:40.509495 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:40.509729 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:40.509815 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:45:40.510071 (+ 256us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 02:45:40.510518 (+ 447us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 02:45:40.510631 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0925 02:45:40.510690 (+ 59us) server_negotiation.cc:299] Negotiation successful
0925 02:45:40.510750 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":77,"threads_started":1}
I20260925 02:45:40.514878 28844 raft_consensus.cc:1261] T 2582c7b01cb645858bebdc8df683334a P 32c471b5298c474ebe1515814e5e6f35 [term 1 FOLLOWER]: Refusing update from remote peer 4ca40898617049bc9142388c6b5c8541: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:45:40.514878 28712 raft_consensus.cc:1261] T 2582c7b01cb645858bebdc8df683334a P 143d3437e4be499fb78ad8ea64dfaaa4 [term 1 FOLLOWER]: Refusing update from remote peer 4ca40898617049bc9142388c6b5c8541: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:45:40.515585 28626 consensus_queue.cc:1035] T 2582c7b01cb645858bebdc8df683334a P 4ca40898617049bc9142388c6b5c8541 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32c471b5298c474ebe1515814e5e6f35" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 40183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:45:40.515759 28911 consensus_queue.cc:1035] T 2582c7b01cb645858bebdc8df683334a P 4ca40898617049bc9142388c6b5c8541 [LEADER]: Connected to new peer: Peer: permanent_uuid: "143d3437e4be499fb78ad8ea64dfaaa4" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 35399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:45:40.524133 28916 mvcc.cc:205] Tried to move back new op lower bound from 7333086578742489088 to 7333086578628427776. Current Snapshot: MvccSnapshot[applied={T|T < 7333086578742489088}]
I20260925 02:45:40.528206 28918 mvcc.cc:205] Tried to move back new op lower bound from 7333086578742489088 to 7333086578628427776. Current Snapshot: MvccSnapshot[applied={T|T < 7333086578742489088}]
I20260925 02:45:40.531070 28431 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:41922:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 02:45:40.531250 28431 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:41922:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 02:45:40.533792 28431 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dc06e289c66345e0b45649408d6da197: Sending DeleteTablet for 3 replicas of tablet 2582c7b01cb645858bebdc8df683334a
I20260925 02:45:40.534394 28824 tablet_service.cc:1514] Processing DeleteTablet for tablet 2582c7b01cb645858bebdc8df683334a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:45:40 UTC) from {username='kudu', principal='kudu/127.13.157.190@KRBTEST.COM'} at 127.0.0.1:51444
I20260925 02:45:40.534583 28554 tablet_service.cc:1514] Processing DeleteTablet for tablet 2582c7b01cb645858bebdc8df683334a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:45:40 UTC) from {username='kudu', principal='kudu/127.13.157.190@KRBTEST.COM'} at 127.0.0.1:60184
I20260925 02:45:40.534948 28925 tablet_replica.cc:330] T 2582c7b01cb645858bebdc8df683334a P 32c471b5298c474ebe1515814e5e6f35: stopping tablet replica
I20260925 02:45:40.534981 28692 tablet_service.cc:1514] Processing DeleteTablet for tablet 2582c7b01cb645858bebdc8df683334a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:45:40 UTC) from {username='kudu', principal='kudu/127.13.157.190@KRBTEST.COM'} at 127.0.0.1:37388
I20260925 02:45:40.535254 28925 raft_consensus.cc:2229] T 2582c7b01cb645858bebdc8df683334a P 32c471b5298c474ebe1515814e5e6f35 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 02:45:40.535542 28925 raft_consensus.cc:2258] T 2582c7b01cb645858bebdc8df683334a P 32c471b5298c474ebe1515814e5e6f35 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:45:40.535637 28927 tablet_replica.cc:330] T 2582c7b01cb645858bebdc8df683334a P 143d3437e4be499fb78ad8ea64dfaaa4: stopping tablet replica
I20260925 02:45:40.535676 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 28470
I20260925 02:45:40.536602 28927 raft_consensus.cc:2229] T 2582c7b01cb645858bebdc8df683334a P 143d3437e4be499fb78ad8ea64dfaaa4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 02:45:40.536831 28927 raft_consensus.cc:2258] T 2582c7b01cb645858bebdc8df683334a P 143d3437e4be499fb78ad8ea64dfaaa4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:45:40.538097 28927 ts_tablet_manager.cc:1905] T 2582c7b01cb645858bebdc8df683334a P 143d3437e4be499fb78ad8ea64dfaaa4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:45:40.538121 28925 ts_tablet_manager.cc:1905] T 2582c7b01cb645858bebdc8df683334a P 32c471b5298c474ebe1515814e5e6f35: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:45:40.540316 28927 ts_tablet_manager.cc:1918] T 2582c7b01cb645858bebdc8df683334a P 143d3437e4be499fb78ad8ea64dfaaa4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 02:45:40.540406 28927 log.cc:1196] T 2582c7b01cb645858bebdc8df683334a P 143d3437e4be499fb78ad8ea64dfaaa4: Deleting WAL directory at /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-1/wal/wals/2582c7b01cb645858bebdc8df683334a
I20260925 02:45:40.540689 28927 ts_tablet_manager.cc:1939] T 2582c7b01cb645858bebdc8df683334a P 143d3437e4be499fb78ad8ea64dfaaa4: Deleting consensus metadata
I20260925 02:45:40.540697 28925 ts_tablet_manager.cc:1918] T 2582c7b01cb645858bebdc8df683334a P 32c471b5298c474ebe1515814e5e6f35: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 02:45:40.540782 28925 log.cc:1196] T 2582c7b01cb645858bebdc8df683334a P 32c471b5298c474ebe1515814e5e6f35: Deleting WAL directory at /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-2/wal/wals/2582c7b01cb645858bebdc8df683334a
I20260925 02:45:40.541064 28925 ts_tablet_manager.cc:1939] T 2582c7b01cb645858bebdc8df683334a P 32c471b5298c474ebe1515814e5e6f35: Deleting consensus metadata
I20260925 02:45:40.541841 28419 catalog_manager.cc:4841] TS 143d3437e4be499fb78ad8ea64dfaaa4 (127.13.157.130:35399): tablet 2582c7b01cb645858bebdc8df683334a (table test-table [id=2831658670ec445c8ea7c21d05e843ed]) successfully deleted
I20260925 02:45:40.542098 28418 catalog_manager.cc:4841] TS 32c471b5298c474ebe1515814e5e6f35 (127.13.157.131:40183): tablet 2582c7b01cb645858bebdc8df683334a (table test-table [id=2831658670ec445c8ea7c21d05e843ed]) successfully deleted
W20260925 02:45:40.543463 28417 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.157.129:45983 (error 108)
I20260925 02:45:40.544243 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 28624
I20260925 02:45:40.544406 28627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:40.544124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.129:45983 (local address 127.0.0.1:60206)
0925 02:45:40.544240 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:40.544302 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.129:45983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260925 02:45:40.545123 28417 catalog_manager.cc:4584] TS 4ca40898617049bc9142388c6b5c8541 (127.13.157.129:45983): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2582c7b01cb645858bebdc8df683334a: Network error: Client connection negotiation failed: client connection to 127.13.157.129:45983: connect: Connection refused (error 111)
I20260925 02:45:40.549831 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 28761
I20260925 02:45:40.555063 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 28398
I20260925 02:45:40.560755 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.157.190:34431
--webserver_interface=127.13.157.190
--webserver_port=43715
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab
--principal=kudu/127.13.157.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:46509
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.157.190:34431
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:45:40.670825 28929 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:45:40.671124 28929 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:45:40.671169 28929 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:45:40.675220 28929 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:45:40.675282 28929 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:45:40.675307 28929 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:45:40.675328 28929 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:45:40.675347 28929 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:45:40.680713 28929 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:46509
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.157.190:34431
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab
--principal=kudu/127.13.157.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.157.190:34431
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.157.190
--webserver_port=43715
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28929
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:45:40.681777 28929 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:45:40.682840 28929 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:45:40.691421 28937 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:45:40.691381 28935 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:45:40.691608 28929 server_base.cc:1034] running on GCE node
W20260925 02:45:40.691403 28934 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:45:40.692253 28929 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:45:40.693358 28929 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:45:40.694526 28929 hybrid_clock.cc:648] HybridClock initialized: now 1790304340694502 us; error 36 us; skew 500 ppm
Sep 25 02:45:40 dist-test-slave-kznw krb5kdc[27847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304340, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:45:40.697268 28929 init.cc:377] Logged in from keytab as kudu/127.13.157.190@KRBTEST.COM (short username kudu)
I20260925 02:45:40.698156 28929 webserver.cc:460] Webserver started at http://127.13.157.190:43715/ using document root <none> and password file <none>
I20260925 02:45:40.698686 28929 fs_manager.cc:359] Metadata directory not provided
I20260925 02:45:40.698746 28929 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:45:40.702557 28929 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260925 02:45:40.704792 28944 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:45:40.706050 28929 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260925 02:45:40.706187 28929 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/master-0/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/master-0/wal
uuid: "dc06e289c66345e0b45649408d6da197"
format_stamp: "Formatted at 2026-09-25 02:45:38 on dist-test-slave-kznw"
server_key: "f97de62a81434f997a2859b500722b3b"
server_key_iv: "2e8faf4b230cedbf5bcde21f4850148c"
server_key_version: "encryptionkey@0"
I20260925 02:45:40.706725 28929 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:45:40.735982 28929 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:45:40.739475 28929 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:45:40.739686 28929 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:45:40.747047 28929 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.190:34431
I20260925 02:45:40.747346 28996 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.190:34431 every 8 connection(s)
I20260925 02:45:40.748317 28929 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
I20260925 02:45:40.752746 28997 sys_catalog.cc:263] Verifying existing consensus state
I20260925 02:45:40.754462 28997 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc06e289c66345e0b45649408d6da197: Bootstrap starting.
I20260925 02:45:40.757341 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 28929
I20260925 02:45:40.757929 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.157.129:45983
--local_ip_for_outbound_sockets=127.13.157.129
--tserver_master_addrs=127.13.157.190:34431
--webserver_port=40749
--webserver_interface=127.13.157.129
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.129.keytab
--principal=kudu/127.13.157.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:46509
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 02:45:40.776378 28997 log.cc:824] T 00000000000000000000000000000000 P dc06e289c66345e0b45649408d6da197: Log is configured to *not* fsync() on all Append() calls
I20260925 02:45:40.790611 28997 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc06e289c66345e0b45649408d6da197: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 02:45:40.791105 28997 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc06e289c66345e0b45649408d6da197: Bootstrap complete.
I20260925 02:45:40.796500 28997 raft_consensus.cc:348] T 00000000000000000000000000000000 P dc06e289c66345e0b45649408d6da197 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc06e289c66345e0b45649408d6da197" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 34431 } }
I20260925 02:45:40.796988 28997 raft_consensus.cc:729] T 00000000000000000000000000000000 P dc06e289c66345e0b45649408d6da197 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc06e289c66345e0b45649408d6da197, State: Initialized, Role: FOLLOWER
I20260925 02:45:40.797540 28997 consensus_queue.cc:260] T 00000000000000000000000000000000 P dc06e289c66345e0b45649408d6da197 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc06e289c66345e0b45649408d6da197" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 34431 } }
I20260925 02:45:40.797689 28997 raft_consensus.cc:388] T 00000000000000000000000000000000 P dc06e289c66345e0b45649408d6da197 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 02:45:40.797765 28997 raft_consensus.cc:482] T 00000000000000000000000000000000 P dc06e289c66345e0b45649408d6da197 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 02:45:40.797861 28997 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dc06e289c66345e0b45649408d6da197 [term 2 FOLLOWER]: Advancing to term 3
I20260925 02:45:40.800453 28997 raft_consensus.cc:504] T 00000000000000000000000000000000 P dc06e289c66345e0b45649408d6da197 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc06e289c66345e0b45649408d6da197" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 34431 } }
I20260925 02:45:40.800817 28997 leader_election.cc:302] T 00000000000000000000000000000000 P dc06e289c66345e0b45649408d6da197 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dc06e289c66345e0b45649408d6da197; no voters:
I20260925 02:45:40.801165 28997 leader_election.cc:288] T 00000000000000000000000000000000 P dc06e289c66345e0b45649408d6da197 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260925 02:45:40.801312 29001 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dc06e289c66345e0b45649408d6da197 [term 3 FOLLOWER]: Leader election won for term 3
I20260925 02:45:40.801555 29001 raft_consensus.cc:686] T 00000000000000000000000000000000 P dc06e289c66345e0b45649408d6da197 [term 3 LEADER]: Becoming Leader. State: Replica: dc06e289c66345e0b45649408d6da197, State: Running, Role: LEADER
I20260925 02:45:40.801875 29001 consensus_queue.cc:237] T 00000000000000000000000000000000 P dc06e289c66345e0b45649408d6da197 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc06e289c66345e0b45649408d6da197" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 34431 } }
I20260925 02:45:40.802436 28997 sys_catalog.cc:564] T 00000000000000000000000000000000 P dc06e289c66345e0b45649408d6da197 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:45:40.803236 29002 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc06e289c66345e0b45649408d6da197 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "dc06e289c66345e0b45649408d6da197" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc06e289c66345e0b45649408d6da197" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 34431 } } }
I20260925 02:45:40.803356 29002 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc06e289c66345e0b45649408d6da197 [sys.catalog]: This master's current role is: LEADER
I20260925 02:45:40.803560 29003 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc06e289c66345e0b45649408d6da197 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dc06e289c66345e0b45649408d6da197. Latest consensus state: current_term: 3 leader_uuid: "dc06e289c66345e0b45649408d6da197" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc06e289c66345e0b45649408d6da197" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 34431 } } }
I20260925 02:45:40.803642 29003 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc06e289c66345e0b45649408d6da197 [sys.catalog]: This master's current role is: LEADER
I20260925 02:45:40.803869 29010 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:45:40.809262 29010 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:45:40.809986 29010 catalog_manager.cc:1259] Loaded cluster ID: 303bcc0b7d1a473e887055648a7ee134
I20260925 02:45:40.810084 29010 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260925 02:45:40.811327 29010 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fd416b5d9ad google::LogMessage::Fail() at ??:0
@ 0x7fd416b6088a google::LogMessage::SendToLog() at ??:0
@ 0x7fd416b5d4de google::LogMessage::Flush() at ??:0
@ 0x7fd416b5ea19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fd42149cfdd _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7fd42149dd2f kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fd42149a579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7fd4214de640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7fd418daf3d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fd418dafc83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7fd418db13f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7fd418da0512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fd41a3e46db start_thread at ??:0
@ 0x7fd41565b61f clone at ??:0
W20260925 02:45:40.910488 28999 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:45:40.910888 28999 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:45:40.910971 28999 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:45:40.917088 28999 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:45:40.917181 28999 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:45:40.917273 28999 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.157.129
I20260925 02:45:40.925264 28999 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:46509
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.129.keytab
--principal=kudu/127.13.157.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.157.129:45983
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.157.129
--webserver_port=40749
--tserver_master_addrs=127.13.157.190:34431
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28999
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.157.129
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:45:40.926667 28999 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:45:40.928308 28999 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:45:40.939823 29040 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:45:40.939831 29039 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:45:40.939831 29042 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:45:40.940572 28999 server_base.cc:1034] running on GCE node
I20260925 02:45:40.941030 28999 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:45:40.941696 28999 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:45:40.942894 28999 hybrid_clock.cc:648] HybridClock initialized: now 1790304340942849 us; error 60 us; skew 500 ppm
Sep 25 02:45:40 dist-test-slave-kznw krb5kdc[27847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304340, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:45:40.945750 28999 init.cc:377] Logged in from keytab as kudu/127.13.157.129@KRBTEST.COM (short username kudu)
I20260925 02:45:40.946659 28999 webserver.cc:460] Webserver started at http://127.13.157.129:40749/ using document root <none> and password file <none>
I20260925 02:45:40.947248 28999 fs_manager.cc:359] Metadata directory not provided
I20260925 02:45:40.947312 28999 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:45:40.951550 28999 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260925 02:45:40.953951 29049 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:45:40.955473 28999 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 02:45:40.955586 28999 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-0/wal
uuid: "4ca40898617049bc9142388c6b5c8541"
format_stamp: "Formatted at 2026-09-25 02:45:38 on dist-test-slave-kznw"
server_key: "64b786325f8a3832ea4e8df017c64fcf"
server_key_iv: "d1c2e18f4d156da8822c91f076411eb6"
server_key_version: "encryptionkey@0"
I20260925 02:45:40.956153 28999 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:45:40.969555 28999 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:45:40.972685 28999 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:45:40.972862 28999 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:45:40.973491 28999 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:45:40.974627 29056 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260925 02:45:40.980270 28999 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260925 02:45:40.980355 28999 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20260925 02:45:40.980425 28999 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260925 02:45:40.982096 28999 ts_tablet_manager.cc:610] Registered 1 tablets
I20260925 02:45:40.982152 28999 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260925 02:45:40.982468 29056 tablet_bootstrap.cc:492] T 2582c7b01cb645858bebdc8df683334a P 4ca40898617049bc9142388c6b5c8541: Bootstrap starting.
I20260925 02:45:40.991946 28999 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.129:45983
I20260925 02:45:40.992219 29165 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.129:45983 every 8 connection(s)
I20260925 02:45:40.992949 28999 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-0/data/info.pb
I20260925 02:45:40.995854 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 28999
I20260925 02:45:40.996240 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.157.130:35399
--local_ip_for_outbound_sockets=127.13.157.130
--tserver_master_addrs=127.13.157.190:34431
--webserver_port=40957
--webserver_interface=127.13.157.130
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.130.keytab
--principal=kudu/127.13.157.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:46509
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 02:45:41.004163 29056 log.cc:824] T 2582c7b01cb645858bebdc8df683334a P 4ca40898617049bc9142388c6b5c8541: Log is configured to *not* fsync() on all Append() calls
I20260925 02:45:41.013576 29056 tablet_bootstrap.cc:492] T 2582c7b01cb645858bebdc8df683334a P 4ca40898617049bc9142388c6b5c8541: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 02:45:41.014040 29056 tablet_bootstrap.cc:492] T 2582c7b01cb645858bebdc8df683334a P 4ca40898617049bc9142388c6b5c8541: Bootstrap complete.
I20260925 02:45:41.015425 29056 ts_tablet_manager.cc:1397] T 2582c7b01cb645858bebdc8df683334a P 4ca40898617049bc9142388c6b5c8541: Time spent bootstrapping tablet: real 0.033s user 0.022s sys 0.010s
I20260925 02:45:41.018859 29056 raft_consensus.cc:348] T 2582c7b01cb645858bebdc8df683334a P 4ca40898617049bc9142388c6b5c8541 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c471b5298c474ebe1515814e5e6f35" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 40183 } } peers { permanent_uuid: "143d3437e4be499fb78ad8ea64dfaaa4" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 35399 } } peers { permanent_uuid: "4ca40898617049bc9142388c6b5c8541" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 45983 } }
I20260925 02:45:41.019477 29056 raft_consensus.cc:729] T 2582c7b01cb645858bebdc8df683334a P 4ca40898617049bc9142388c6b5c8541 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ca40898617049bc9142388c6b5c8541, State: Initialized, Role: FOLLOWER
I20260925 02:45:41.020107 29056 consensus_queue.cc:260] T 2582c7b01cb645858bebdc8df683334a P 4ca40898617049bc9142388c6b5c8541 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c471b5298c474ebe1515814e5e6f35" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 40183 } } peers { permanent_uuid: "143d3437e4be499fb78ad8ea64dfaaa4" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 35399 } } peers { permanent_uuid: "4ca40898617049bc9142388c6b5c8541" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 45983 } }
I20260925 02:45:41.021199 29056 ts_tablet_manager.cc:1428] T 2582c7b01cb645858bebdc8df683334a P 4ca40898617049bc9142388c6b5c8541: Time spent starting tablet: real 0.006s user 0.005s sys 0.001s
W20260925 02:45:41.046725 29168 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:45:40.994822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:34431 (local address 127.13.157.129:34939)
0925 02:45:40.995336 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:40.995389 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0925 02:45:40.997271 (+ 1882us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:45:41.045959 (+ 48688us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:34431: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":119,"threads_started":1}
W20260925 02:45:41.047235 29166 heartbeater.cc:644] Failed to heartbeat to 127.13.157.190:34431 (0 consecutive failures): Network error: Failed to ping master at 127.13.157.190:34431: Client connection negotiation failed: client connection to 127.13.157.190:34431: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260925 02:45:41.048012 29168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:41.047737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:34431 (local address 127.13.157.129:50183)
0925 02:45:41.047882 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:41.047950 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:34431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260925 02:45:41.048689 29168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:41.048486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:34431 (local address 127.13.157.129:57909)
0925 02:45:41.048594 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:41.048638 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:34431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260925 02:45:41.048836 29166 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260925 02:45:41.109768 29169 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:45:41.110056 29169 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:45:41.110121 29169 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:45:41.114287 29169 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:45:41.114352 29169 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:45:41.114434 29169 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.157.130
I20260925 02:45:41.119828 29169 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:46509
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.130.keytab
--principal=kudu/127.13.157.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.157.130:35399
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.157.130
--webserver_port=40957
--tserver_master_addrs=127.13.157.190:34431
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29169
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.157.130
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:45:41.120954 29169 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:45:41.122046 29169 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:45:41.131605 29176 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:45:41.131654 29179 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:45:41.131634 29177 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:45:41.131866 29169 server_base.cc:1034] running on GCE node
I20260925 02:45:41.132331 29169 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:45:41.132973 29169 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:45:41.134124 29169 hybrid_clock.cc:648] HybridClock initialized: now 1790304341134099 us; error 37 us; skew 500 ppm
Sep 25 02:45:41 dist-test-slave-kznw krb5kdc[27847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304341, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:45:41.136816 29169 init.cc:377] Logged in from keytab as kudu/127.13.157.130@KRBTEST.COM (short username kudu)
I20260925 02:45:41.137686 29169 webserver.cc:460] Webserver started at http://127.13.157.130:40957/ using document root <none> and password file <none>
I20260925 02:45:41.138208 29169 fs_manager.cc:359] Metadata directory not provided
I20260925 02:45:41.138274 29169 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:45:41.142040 29169 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260925 02:45:41.144109 29186 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:45:41.145401 29169 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260925 02:45:41.145519 29169 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-1/wal
uuid: "143d3437e4be499fb78ad8ea64dfaaa4"
format_stamp: "Formatted at 2026-09-25 02:45:39 on dist-test-slave-kznw"
server_key: "9b52e7135c42992013322d738375c784"
server_key_iv: "c18cb6d8e6c95eb58062bb8b1be47467"
server_key_version: "encryptionkey@0"
I20260925 02:45:41.146057 29169 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:45:41.162667 29169 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:45:41.165840 29169 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:45:41.166026 29169 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:45:41.166636 29169 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:45:41.167506 29169 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:45:41.167555 29169 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:45:41.167609 29169 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:45:41.167637 29169 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:45:41.176215 29169 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.130:35399
I20260925 02:45:41.176301 29299 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.130:35399 every 8 connection(s)
I20260925 02:45:41.177222 29169 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-1/data/info.pb
I20260925 02:45:41.180991 29302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:41.179356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:34431 (local address 127.13.157.130:37601)
0925 02:45:41.179862 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:41.180277 (+ 415us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:34431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":104,"threads_started":1}
W20260925 02:45:41.181447 29300 heartbeater.cc:644] Failed to heartbeat to 127.13.157.190:34431 (0 consecutive failures): Network error: Failed to ping master at 127.13.157.190:34431: Client connection negotiation failed: client connection to 127.13.157.190:34431: connect: Connection refused (error 111)
I20260925 02:45:41.181978 29302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:41.181772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:34431 (local address 127.13.157.130:52073)
0925 02:45:41.181879 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:41.181924 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:34431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260925 02:45:41.182540 29302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:41.182351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:34431 (local address 127.13.157.130:42377)
0925 02:45:41.182448 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:41.182491 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:34431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260925 02:45:41.182684 29300 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260925 02:45:41.182988 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 29169
I20260925 02:45:41.183346 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.157.131:40183
--local_ip_for_outbound_sockets=127.13.157.131
--tserver_master_addrs=127.13.157.190:34431
--webserver_port=35755
--webserver_interface=127.13.157.131
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.131.keytab
--principal=kudu/127.13.157.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:46509
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:45:41.293270 29303 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:45:41.293556 29303 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:45:41.293609 29303 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:45:41.297791 29303 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:45:41.297869 29303 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:45:41.297930 29303 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.157.131
I20260925 02:45:41.303507 29303 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:46509
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.131.keytab
--principal=kudu/127.13.157.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.157.131:40183
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.157.131
--webserver_port=35755
--tserver_master_addrs=127.13.157.190:34431
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29303
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.157.131
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:45:41.304584 29303 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:45:41.305588 29303 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:45:41.315176 29311 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:45:41.315251 29308 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:45:41.315291 29309 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:45:41.315546 29303 server_base.cc:1034] running on GCE node
I20260925 02:45:41.315955 29303 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:45:41.316557 29303 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:45:41.317745 29303 hybrid_clock.cc:648] HybridClock initialized: now 1790304341317728 us; error 41 us; skew 500 ppm
Sep 25 02:45:41 dist-test-slave-kznw krb5kdc[27847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304341, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:45:41.320477 29303 init.cc:377] Logged in from keytab as kudu/127.13.157.131@KRBTEST.COM (short username kudu)
I20260925 02:45:41.321427 29303 webserver.cc:460] Webserver started at http://127.13.157.131:35755/ using document root <none> and password file <none>
I20260925 02:45:41.321969 29303 fs_manager.cc:359] Metadata directory not provided
I20260925 02:45:41.322036 29303 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:45:41.325891 29303 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260925 02:45:41.328249 29318 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:45:41.329555 29303 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260925 02:45:41.329705 29303 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-2/wal
uuid: "32c471b5298c474ebe1515814e5e6f35"
format_stamp: "Formatted at 2026-09-25 02:45:39 on dist-test-slave-kznw"
server_key: "97d2bc30a81a3deef3cda089102dc495"
server_key_iv: "e5cedfb7525abc455d8d832ac3cd7571"
server_key_version: "encryptionkey@0"
I20260925 02:45:41.330258 29303 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:45:41.343046 29303 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:45:41.346010 29303 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:45:41.346204 29303 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:45:41.346832 29303 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:45:41.347752 29303 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:45:41.347815 29303 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:45:41.347872 29303 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:45:41.347901 29303 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:45:41.357265 29303 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.131:40183
I20260925 02:45:41.357293 29431 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.131:40183 every 8 connection(s)
I20260925 02:45:41.358336 29303 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790304178325258-13942-0/minicluster-data/ts-2/data/info.pb
I20260925 02:45:41.359319 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 29303
I20260925 02:45:41.362491 29435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:41.360818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:34431 (local address 127.13.157.131:42267)
0925 02:45:41.361295 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:41.361727 (+ 432us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:34431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":112,"threads_started":1}
W20260925 02:45:41.362962 29432 heartbeater.cc:644] Failed to heartbeat to 127.13.157.190:34431 (0 consecutive failures): Network error: Failed to ping master at 127.13.157.190:34431: Client connection negotiation failed: client connection to 127.13.157.190:34431: connect: Connection refused (error 111)
I20260925 02:45:41.363539 29435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:41.363334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:34431 (local address 127.13.157.131:33661)
0925 02:45:41.363434 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:41.363483 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:34431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260925 02:45:41.364153 29435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:41.363906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:34431 (local address 127.13.157.131:45257)
0925 02:45:41.364040 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:41.364087 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:34431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260925 02:45:41.364353 29432 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260925 02:45:42.050010 29436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:42.049429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:34431 (local address 127.13.157.129:49791)
0925 02:45:42.049808 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:42.049884 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:34431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":78,"threads_started":1}
I20260925 02:45:42.183763 29437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:42.183260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:34431 (local address 127.13.157.130:33701)
0925 02:45:42.183534 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:42.183623 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:34431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":68,"threads_started":1}
I20260925 02:45:42.365335 29438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:42.364888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:34431 (local address 127.13.157.131:58483)
0925 02:45:42.365157 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:42.365227 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:34431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":69,"threads_started":1}
I20260925 02:45:42.681511 29439 raft_consensus.cc:482] T 2582c7b01cb645858bebdc8df683334a P 4ca40898617049bc9142388c6b5c8541 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 02:45:42.681777 29439 raft_consensus.cc:504] T 2582c7b01cb645858bebdc8df683334a P 4ca40898617049bc9142388c6b5c8541 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c471b5298c474ebe1515814e5e6f35" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 40183 } } peers { permanent_uuid: "143d3437e4be499fb78ad8ea64dfaaa4" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 35399 } } peers { permanent_uuid: "4ca40898617049bc9142388c6b5c8541" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 45983 } }
I20260925 02:45:42.682844 29439 leader_election.cc:288] T 2582c7b01cb645858bebdc8df683334a P 4ca40898617049bc9142388c6b5c8541 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 32c471b5298c474ebe1515814e5e6f35 (127.13.157.131:40183), 143d3437e4be499fb78ad8ea64dfaaa4 (127.13.157.130:35399)
Sep 25 02:45:42 dist-test-slave-kznw krb5kdc[27847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304340, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.129@KRBTEST.COM for kudu/127.13.157.131@KRBTEST.COM
Sep 25 02:45:42 dist-test-slave-kznw krb5kdc[27847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304340, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.129@KRBTEST.COM for kudu/127.13.157.130@KRBTEST.COM
I20260925 02:45:42.696698 29441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:42.683058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.130:35399 (local address 127.13.157.129:49997)
0925 02:45:42.683373 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:42.683388 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 02:45:42.683502 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:45:42.685115 (+ 1613us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:45:42.685121 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:45:42.685623 (+ 502us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:45:42.686119 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:45:42.686134 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:45:42.687399 (+ 1265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:45:42.687401 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:45:42.688334 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:45:42.688343 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:45:42.689002 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:45:42.689005 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:45:42.689243 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:45:42.689851 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:45:42.689863 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:45:42.691681 (+ 1818us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:45:42.693805 (+ 2124us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:45:42.693811 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:45:42.693826 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:45:42.694256 (+ 430us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:45:42.694625 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:45:42.694628 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:45:42.694631 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:45:42.694844 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:45:42.695604 (+ 760us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:45:42.695611 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:45:42.695946 (+ 335us) client_negotiation.cc:769] Sending connection context
0925 02:45:42.696170 (+ 224us) client_negotiation.cc:240] Negotiation successful
0925 02:45:42.696488 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"mutex_wait_us":216,"spinlock_wait_cycles":20096,"thread_start_us":77,"threads_started":1}
I20260925 02:45:42.696789 29443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:42.683317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.129:49997 (local address 127.13.157.130:35399)
0925 02:45:42.683706 (+ 389us) server_negotiation.cc:206] Beginning negotiation
0925 02:45:42.683732 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0925 02:45:42.683771 (+ 39us) server_negotiation.cc:407] Connection header received
0925 02:45:42.684447 (+ 676us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:45:42.684453 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:45:42.684769 (+ 316us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:45:42.685038 (+ 269us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:45:42.686275 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:42.687284 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:42.688449 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:42.688886 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:42.689027 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:45:42.691872 (+ 2845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:45:42.691879 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:45:42.691892 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:45:42.691922 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:45:42.693710 (+ 1788us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:45:42.694372 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:45:42.694377 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:45:42.694383 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:45:42.694533 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:45:42.694964 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:45:42.694967 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:45:42.694969 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:45:42.695488 (+ 519us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:45:42.695616 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0925 02:45:42.696298 (+ 682us) server_negotiation.cc:299] Negotiation successful
0925 02:45:42.696580 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":266,"thread_start_us":75,"threads_started":1}
I20260925 02:45:42.696805 29442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:42.683290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.129:39519 (local address 127.13.157.131:40183)
0925 02:45:42.683709 (+ 419us) server_negotiation.cc:206] Beginning negotiation
0925 02:45:42.683731 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0925 02:45:42.683770 (+ 39us) server_negotiation.cc:407] Connection header received
0925 02:45:42.684432 (+ 662us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:45:42.684437 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:45:42.684769 (+ 332us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:45:42.685039 (+ 270us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:45:42.686283 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:42.687264 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:42.688450 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:42.688885 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:42.689026 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:45:42.691701 (+ 2675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:45:42.691708 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:45:42.691721 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:45:42.691755 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:45:42.693674 (+ 1919us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:45:42.694371 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:45:42.694377 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:45:42.694383 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:45:42.694537 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:45:42.694964 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:45:42.694967 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:45:42.694969 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:45:42.695488 (+ 519us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:45:42.695675 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0925 02:45:42.696298 (+ 623us) server_negotiation.cc:299] Negotiation successful
0925 02:45:42.696580 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":57,"threads_started":1}
I20260925 02:45:42.696709 29440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:42.683058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.131:40183 (local address 127.13.157.129:39519)
0925 02:45:42.683496 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:42.683510 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 02:45:42.683651 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:45:42.685116 (+ 1465us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:45:42.685122 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:45:42.685628 (+ 506us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:45:42.686120 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:45:42.686134 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:45:42.687394 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:45:42.687397 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:45:42.688336 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:45:42.688346 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:45:42.689001 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:45:42.689005 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:45:42.689242 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:45:42.689815 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:45:42.689830 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:45:42.691376 (+ 1546us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:45:42.693805 (+ 2429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:45:42.693811 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:45:42.693826 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:45:42.694256 (+ 430us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:45:42.694625 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:45:42.694628 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:45:42.694630 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:45:42.694844 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:45:42.695604 (+ 760us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:45:42.695611 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:45:42.695946 (+ 335us) client_negotiation.cc:769] Sending connection context
0925 02:45:42.696170 (+ 224us) client_negotiation.cc:240] Negotiation successful
0925 02:45:42.696550 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":366,"mutex_wait_us":409,"spinlock_wait_cycles":45440,"thread_start_us":76,"threads_started":1}
W20260925 02:45:42.699213 29052 leader_election.cc:341] T 2582c7b01cb645858bebdc8df683334a P 4ca40898617049bc9142388c6b5c8541 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 32c471b5298c474ebe1515814e5e6f35 (127.13.157.131:40183): Not found: Tablet not found: 2582c7b01cb645858bebdc8df683334a
W20260925 02:45:42.699380 29053 leader_election.cc:341] T 2582c7b01cb645858bebdc8df683334a P 4ca40898617049bc9142388c6b5c8541 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 143d3437e4be499fb78ad8ea64dfaaa4 (127.13.157.130:35399): Not found: Tablet not found: 2582c7b01cb645858bebdc8df683334a
I20260925 02:45:42.699451 29053 leader_election.cc:302] T 2582c7b01cb645858bebdc8df683334a P 4ca40898617049bc9142388c6b5c8541 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4ca40898617049bc9142388c6b5c8541; no voters: 143d3437e4be499fb78ad8ea64dfaaa4, 32c471b5298c474ebe1515814e5e6f35
I20260925 02:45:42.699726 29439 raft_consensus.cc:2726] T 2582c7b01cb645858bebdc8df683334a P 4ca40898617049bc9142388c6b5c8541 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 02:45:43.051172 29440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:43.050819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:34431 (local address 127.13.157.129:41869)
0925 02:45:43.050994 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:43.051059 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:34431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260925 02:45:43.185117 29444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:43.184577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:34431 (local address 127.13.157.130:51117)
0925 02:45:43.184909 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:43.184983 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:34431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":76,"threads_started":1}
I20260925 02:45:43.366493 29445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:43.366089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:34431 (local address 127.13.157.131:59431)
0925 02:45:43.366346 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:43.366403 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:34431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":73,"threads_started":1}
I20260925 02:45:44.052277 29446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:44.051846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:34431 (local address 127.13.157.129:45777)
0925 02:45:44.052110 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:44.052166 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:34431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":75,"threads_started":1}
I20260925 02:45:44.186357 29447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:44.185894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:34431 (local address 127.13.157.130:44635)
0925 02:45:44.186190 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:44.186255 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:34431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":86,"threads_started":1}
I20260925 02:45:44.367771 29448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:44.367287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:34431 (local address 127.13.157.131:54125)
0925 02:45:44.367557 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:44.367622 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:34431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":77,"threads_started":1}
I20260925 02:45:44.930691 29449 raft_consensus.cc:482] T 2582c7b01cb645858bebdc8df683334a P 4ca40898617049bc9142388c6b5c8541 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 02:45:44.930889 29449 raft_consensus.cc:504] T 2582c7b01cb645858bebdc8df683334a P 4ca40898617049bc9142388c6b5c8541 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c471b5298c474ebe1515814e5e6f35" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 40183 } } peers { permanent_uuid: "143d3437e4be499fb78ad8ea64dfaaa4" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 35399 } } peers { permanent_uuid: "4ca40898617049bc9142388c6b5c8541" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 45983 } }
I20260925 02:45:44.931339 29449 leader_election.cc:288] T 2582c7b01cb645858bebdc8df683334a P 4ca40898617049bc9142388c6b5c8541 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 32c471b5298c474ebe1515814e5e6f35 (127.13.157.131:40183), 143d3437e4be499fb78ad8ea64dfaaa4 (127.13.157.130:35399)
W20260925 02:45:44.932147 29053 leader_election.cc:341] T 2582c7b01cb645858bebdc8df683334a P 4ca40898617049bc9142388c6b5c8541 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 143d3437e4be499fb78ad8ea64dfaaa4 (127.13.157.130:35399): Not found: Tablet not found: 2582c7b01cb645858bebdc8df683334a
W20260925 02:45:44.932312 29052 leader_election.cc:341] T 2582c7b01cb645858bebdc8df683334a P 4ca40898617049bc9142388c6b5c8541 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 32c471b5298c474ebe1515814e5e6f35 (127.13.157.131:40183): Not found: Tablet not found: 2582c7b01cb645858bebdc8df683334a
I20260925 02:45:44.932423 29052 leader_election.cc:302] T 2582c7b01cb645858bebdc8df683334a P 4ca40898617049bc9142388c6b5c8541 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4ca40898617049bc9142388c6b5c8541; no voters: 143d3437e4be499fb78ad8ea64dfaaa4, 32c471b5298c474ebe1515814e5e6f35
I20260925 02:45:44.932606 29449 raft_consensus.cc:2726] T 2582c7b01cb645858bebdc8df683334a P 4ca40898617049bc9142388c6b5c8541 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 02:45:45.053529 29450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:45.053067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:34431 (local address 127.13.157.129:48199)
0925 02:45:45.053357 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:45.053424 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:34431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":79,"threads_started":1}
I20260925 02:45:45.187644 29451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:45.187213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:34431 (local address 127.13.157.130:58689)
0925 02:45:45.187466 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:45.187526 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:34431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":73,"threads_started":1}
I20260925 02:45:45.368986 29452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:45.368562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:34431 (local address 127.13.157.131:58711)
0925 02:45:45.368834 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:45.368895 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:34431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":91,"threads_started":1}
I20260925 02:45:46.055027 29453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:46.054473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:34431 (local address 127.13.157.129:43479)
0925 02:45:46.054824 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:46.054888 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:34431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":73,"threads_started":1}
I20260925 02:45:46.188901 29454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:46.188425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:34431 (local address 127.13.157.130:42653)
0925 02:45:46.188739 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:46.188802 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:34431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":74,"threads_started":1}
I20260925 02:45:46.370189 29455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:46.369754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:34431 (local address 127.13.157.131:54225)
0925 02:45:46.370035 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:46.370093 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:34431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":93,"threads_started":1}
I20260925 02:45:47.056123 29456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:47.055714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:34431 (local address 127.13.157.129:55171)
0925 02:45:47.055967 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:47.056024 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:34431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":73,"threads_started":1}
I20260925 02:45:47.190054 29457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:47.189596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:34431 (local address 127.13.157.130:38041)
0925 02:45:47.189889 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:47.189952 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:34431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":92,"threads_started":1}
I20260925 02:45:47.371459 29458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:47.371011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:34431 (local address 127.13.157.131:55927)
0925 02:45:47.371293 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:47.371357 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:34431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":78,"threads_started":1}
I20260925 02:45:47.540675 29459 raft_consensus.cc:482] T 2582c7b01cb645858bebdc8df683334a P 4ca40898617049bc9142388c6b5c8541 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 02:45:47.540804 29459 raft_consensus.cc:504] T 2582c7b01cb645858bebdc8df683334a P 4ca40898617049bc9142388c6b5c8541 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c471b5298c474ebe1515814e5e6f35" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 40183 } } peers { permanent_uuid: "143d3437e4be499fb78ad8ea64dfaaa4" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 35399 } } peers { permanent_uuid: "4ca40898617049bc9142388c6b5c8541" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 45983 } }
I20260925 02:45:47.541216 29459 leader_election.cc:288] T 2582c7b01cb645858bebdc8df683334a P 4ca40898617049bc9142388c6b5c8541 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 32c471b5298c474ebe1515814e5e6f35 (127.13.157.131:40183), 143d3437e4be499fb78ad8ea64dfaaa4 (127.13.157.130:35399)
W20260925 02:45:47.541957 29053 leader_election.cc:341] T 2582c7b01cb645858bebdc8df683334a P 4ca40898617049bc9142388c6b5c8541 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 143d3437e4be499fb78ad8ea64dfaaa4 (127.13.157.130:35399): Not found: Tablet not found: 2582c7b01cb645858bebdc8df683334a
W20260925 02:45:47.542068 29052 leader_election.cc:341] T 2582c7b01cb645858bebdc8df683334a P 4ca40898617049bc9142388c6b5c8541 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 32c471b5298c474ebe1515814e5e6f35 (127.13.157.131:40183): Not found: Tablet not found: 2582c7b01cb645858bebdc8df683334a
I20260925 02:45:47.542160 29052 leader_election.cc:302] T 2582c7b01cb645858bebdc8df683334a P 4ca40898617049bc9142388c6b5c8541 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4ca40898617049bc9142388c6b5c8541; no voters: 143d3437e4be499fb78ad8ea64dfaaa4, 32c471b5298c474ebe1515814e5e6f35
I20260925 02:45:47.542302 29459 raft_consensus.cc:2726] T 2582c7b01cb645858bebdc8df683334a P 4ca40898617049bc9142388c6b5c8541 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 02:45:48.057400 29460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:48.056948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:34431 (local address 127.13.157.129:53185)
0925 02:45:48.057237 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:48.057298 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:34431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":91,"threads_started":1}
I20260925 02:45:48.191318 29461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:48.190837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:34431 (local address 127.13.157.130:55531)
0925 02:45:48.191154 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:48.191206 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:34431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":122,"threads_started":1}
I20260925 02:45:48.372723 29462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:48.372241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:34431 (local address 127.13.157.131:59479)
0925 02:45:48.372562 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:48.372630 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:34431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":70,"threads_started":1}
I20260925 02:45:49.058780 29463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:49.058287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:34431 (local address 127.13.157.129:49667)
0925 02:45:49.058574 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:49.058647 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:34431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":80,"threads_started":1}
I20260925 02:45:49.192551 29464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:49.192060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:34431 (local address 127.13.157.130:51711)
0925 02:45:49.192379 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:49.192446 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:34431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":88,"threads_started":1}
I20260925 02:45:49.249034 29465 raft_consensus.cc:482] T 2582c7b01cb645858bebdc8df683334a P 4ca40898617049bc9142388c6b5c8541 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 02:45:49.249190 29465 raft_consensus.cc:504] T 2582c7b01cb645858bebdc8df683334a P 4ca40898617049bc9142388c6b5c8541 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c471b5298c474ebe1515814e5e6f35" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 40183 } } peers { permanent_uuid: "143d3437e4be499fb78ad8ea64dfaaa4" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 35399 } } peers { permanent_uuid: "4ca40898617049bc9142388c6b5c8541" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 45983 } }
I20260925 02:45:49.249600 29465 leader_election.cc:288] T 2582c7b01cb645858bebdc8df683334a P 4ca40898617049bc9142388c6b5c8541 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 32c471b5298c474ebe1515814e5e6f35 (127.13.157.131:40183), 143d3437e4be499fb78ad8ea64dfaaa4 (127.13.157.130:35399)
W20260925 02:45:49.250381 29053 leader_election.cc:341] T 2582c7b01cb645858bebdc8df683334a P 4ca40898617049bc9142388c6b5c8541 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 143d3437e4be499fb78ad8ea64dfaaa4 (127.13.157.130:35399): Not found: Tablet not found: 2582c7b01cb645858bebdc8df683334a
W20260925 02:45:49.250527 29052 leader_election.cc:341] T 2582c7b01cb645858bebdc8df683334a P 4ca40898617049bc9142388c6b5c8541 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 32c471b5298c474ebe1515814e5e6f35 (127.13.157.131:40183): Not found: Tablet not found: 2582c7b01cb645858bebdc8df683334a
I20260925 02:45:49.250644 29052 leader_election.cc:302] T 2582c7b01cb645858bebdc8df683334a P 4ca40898617049bc9142388c6b5c8541 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4ca40898617049bc9142388c6b5c8541; no voters: 143d3437e4be499fb78ad8ea64dfaaa4, 32c471b5298c474ebe1515814e5e6f35
I20260925 02:45:49.250797 29465 raft_consensus.cc:2726] T 2582c7b01cb645858bebdc8df683334a P 4ca40898617049bc9142388c6b5c8541 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 02:45:49.373849 29466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:49.373425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:34431 (local address 127.13.157.131:56619)
0925 02:45:49.373693 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:49.373758 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:34431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":96,"threads_started":1}
I20260925 02:45:50.060050 29467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:50.059599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:34431 (local address 127.13.157.129:50665)
0925 02:45:50.059887 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:50.059947 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:34431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":74,"threads_started":1}
I20260925 02:45:50.193780 29468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:50.193299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:34431 (local address 127.13.157.130:43375)
0925 02:45:50.193598 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:50.193673 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:34431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":71,"threads_started":1}
I20260925 02:45:50.375084 29469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:50.374609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:34431 (local address 127.13.157.131:44013)
0925 02:45:50.374888 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:50.374985 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:34431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":76,"threads_started":1}
I20260925 02:45:51.061393 29470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:51.060935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:34431 (local address 127.13.157.129:49085)
0925 02:45:51.061223 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:51.061294 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:34431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":77,"threads_started":1}
I20260925 02:45:51.195030 29471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:51.194533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:34431 (local address 127.13.157.130:60501)
0925 02:45:51.194848 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:51.194926 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:34431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":117,"threads_started":1}
I20260925 02:45:51.376286 29472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:51.375852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:34431 (local address 127.13.157.131:50129)
0925 02:45:51.376109 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:51.376187 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:34431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":80,"threads_started":1}
I20260925 02:45:52.062656 29473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:52.062157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:34431 (local address 127.13.157.129:60115)
0925 02:45:52.062422 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:52.062487 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:34431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":78,"threads_started":1}
I20260925 02:45:52.196221 29474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:52.195785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:34431 (local address 127.13.157.130:36303)
0925 02:45:52.196047 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:52.196111 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:34431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":74,"threads_started":1}
I20260925 02:45:52.377540 29475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:52.377121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:34431 (local address 127.13.157.131:54747)
0925 02:45:52.377383 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:52.377445 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:34431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":84,"threads_started":1}
I20260925 02:45:53.063905 29476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:53.063411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:34431 (local address 127.13.157.129:47477)
0925 02:45:53.063747 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:53.063808 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:34431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":83,"threads_started":1}
I20260925 02:45:53.197402 29477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:53.196952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:34431 (local address 127.13.157.130:41443)
0925 02:45:53.197241 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:53.197301 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:34431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":90,"threads_started":1}
I20260925 02:45:53.378679 29478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:53.378265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:34431 (local address 127.13.157.131:48817)
0925 02:45:53.378515 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:53.378572 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:34431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":88,"threads_started":1}
I20260925 02:45:54.065099 29479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:54.064638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:34431 (local address 127.13.157.129:39445)
0925 02:45:54.064934 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:54.064997 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:34431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":76,"threads_started":1}
I20260925 02:45:54.198650 29480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:54.198160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:34431 (local address 127.13.157.130:60669)
0925 02:45:54.198438 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:54.198500 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:34431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":75,"threads_started":1}
I20260925 02:45:54.379961 29481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:54.379464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:34431 (local address 127.13.157.131:34969)
0925 02:45:54.379797 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:54.379858 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:34431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":146,"threads_started":1}
I20260925 02:45:55.066278 29482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:55.065832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:34431 (local address 127.13.157.129:55013)
0925 02:45:55.066106 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:55.066172 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:34431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":78,"threads_started":1}
I20260925 02:45:55.199870 29483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:55.199450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:34431 (local address 127.13.157.130:48369)
0925 02:45:55.199716 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:55.199773 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:34431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":81,"threads_started":1}
I20260925 02:45:55.381161 29484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:55.380753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:34431 (local address 127.13.157.131:50349)
0925 02:45:55.381006 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:55.381064 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:34431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":75,"threads_started":1}
I20260925 02:45:56.067656 29485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:56.067203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:34431 (local address 127.13.157.129:45235)
0925 02:45:56.067460 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:56.067523 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:34431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":73,"threads_started":1}
I20260925 02:45:56.201251 29486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:56.200816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:34431 (local address 127.13.157.130:57427)
0925 02:45:56.201086 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:56.201144 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:34431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":72,"threads_started":1}
I20260925 02:45:56.362054 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 28999
I20260925 02:45:56.367678 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 29169
I20260925 02:45:56.374729 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 29303
2026-09-25T02:45:56Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18950 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 02:45:56 dist-test-slave-kznw krb5kdc[29489](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 02:45:56 dist-test-slave-kznw krb5kdc[29489](info): set up 2 sockets
Sep 25 02:45:56 dist-test-slave-kznw krb5kdc[29489](info): commencing operation
krb5kdc: starting...
W20260925 02:45:57.334578 13942 mini_kdc.cc:121] Time spent starting KDC: real 0.939s user 0.006s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/krb5kdc/test-user.keytab.
Sep 25 02:45:57 dist-test-slave-kznw krb5kdc[29489](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304357, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T02:45:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T02:45:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.157.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.157.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.157.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab.
Entry for principal kudu/127.13.157.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab.
I20260925 02:45:57.451807 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.157.190:41669
--webserver_interface=127.13.157.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab
--principal=kudu/127.13.157.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:38239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.157.190:41669
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:45:57.560822 29502 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:45:57.561098 29502 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:45:57.561141 29502 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:45:57.565135 29502 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260925 02:45:57.565187 29502 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260925 02:45:57.565204 29502 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260925 02:45:57.565220 29502 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:45:57.565243 29502 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:45:57.565282 29502 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:45:57.565320 29502 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:45:57.565352 29502 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:45:57.570605 29502 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:38239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.13.157.190:41669
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab
--principal=kudu/127.13.157.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.157.190:41669
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.157.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29502
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:45:57.571678 29502 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:45:57.572700 29502 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:45:57.581557 29508 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:45:57.581633 29507 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:45:57.581627 29502 server_base.cc:1034] running on GCE node
W20260925 02:45:57.581848 29510 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:45:57.582351 29502 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:45:57.583493 29502 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:45:57.584662 29502 hybrid_clock.cc:648] HybridClock initialized: now 1790304357584645 us; error 29 us; skew 500 ppm
Sep 25 02:45:57 dist-test-slave-kznw krb5kdc[29489](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304357, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:45:57.587281 29502 init.cc:377] Logged in from keytab as kudu/127.13.157.190@KRBTEST.COM (short username kudu)
I20260925 02:45:57.588109 29502 webserver.cc:460] Webserver started at http://127.13.157.190:34797/ using document root <none> and password file <none>
I20260925 02:45:57.588630 29502 fs_manager.cc:359] Metadata directory not provided
I20260925 02:45:57.588693 29502 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:45:57.588892 29502 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:45:57.590873 29502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/master-0/data/instance:
uuid: "350851ecf2bf4219bc7578b7de80ffc1"
format_stamp: "Formatted at 2026-09-25 02:45:57 on dist-test-slave-kznw"
server_key: "c3fe0d9e324b624e722df0418a656b85"
server_key_iv: "f23e3496dfb85ca0bcd7e73cb0a7c965"
server_key_version: "encryptionkey@0"
I20260925 02:45:57.591434 29502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/master-0/wal/instance:
uuid: "350851ecf2bf4219bc7578b7de80ffc1"
format_stamp: "Formatted at 2026-09-25 02:45:57 on dist-test-slave-kznw"
server_key: "c3fe0d9e324b624e722df0418a656b85"
server_key_iv: "f23e3496dfb85ca0bcd7e73cb0a7c965"
server_key_version: "encryptionkey@0"
I20260925 02:45:57.594964 29502 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260925 02:45:57.597210 29517 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:45:57.598371 29502 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260925 02:45:57.598477 29502 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/master-0/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/master-0/wal
uuid: "350851ecf2bf4219bc7578b7de80ffc1"
format_stamp: "Formatted at 2026-09-25 02:45:57 on dist-test-slave-kznw"
server_key: "c3fe0d9e324b624e722df0418a656b85"
server_key_iv: "f23e3496dfb85ca0bcd7e73cb0a7c965"
server_key_version: "encryptionkey@0"
I20260925 02:45:57.598563 29502 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:45:57.623247 29502 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:45:57.626104 29502 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:45:57.626297 29502 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:45:57.633378 29502 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.190:41669
I20260925 02:45:57.633709 29569 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.190:41669 every 8 connection(s)
I20260925 02:45:57.634336 29502 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
I20260925 02:45:57.637130 29570 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:45:57.637838 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 29502
I20260925 02:45:57.637956 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/master-0/wal/instance
I20260925 02:45:57.638247 13942 external_mini_cluster.cc:1428] Setting key e9d427b4186148645807da6ba04f41af
I20260925 02:45:57.644018 29570 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 350851ecf2bf4219bc7578b7de80ffc1: Bootstrap starting.
I20260925 02:45:57.646607 29570 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 350851ecf2bf4219bc7578b7de80ffc1: Neither blocks nor log segments found. Creating new log.
I20260925 02:45:57.647423 29570 log.cc:824] T 00000000000000000000000000000000 P 350851ecf2bf4219bc7578b7de80ffc1: Log is configured to *not* fsync() on all Append() calls
Sep 25 02:45:57 dist-test-slave-kznw krb5kdc[29489](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304357, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.157.190@KRBTEST.COM
I20260925 02:45:57.649735 29570 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 350851ecf2bf4219bc7578b7de80ffc1: No bootstrap required, opened a new log
I20260925 02:45:57.653474 29573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:57.639658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39678 (local address 127.13.157.190:41669)
0925 02:45:57.640104 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0925 02:45:57.640125 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0925 02:45:57.640170 (+ 45us) server_negotiation.cc:407] Connection header received
0925 02:45:57.641005 (+ 835us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:45:57.641031 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:45:57.641483 (+ 452us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:45:57.641871 (+ 388us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:45:57.642516 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:57.644009 (+ 1493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:57.645052 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:57.645792 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:57.645955 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:45:57.648602 (+ 2647us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:45:57.648607 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:45:57.648622 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:45:57.648650 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:45:57.650577 (+ 1927us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:45:57.651079 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:45:57.651084 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:45:57.651089 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:45:57.651218 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:45:57.651488 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:45:57.651490 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:45:57.651491 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:45:57.651914 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:45:57.652097 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0925 02:45:57.652393 (+ 296us) server_negotiation.cc:299] Negotiation successful
0925 02:45:57.652679 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":119,"threads_started":1}
I20260925 02:45:57.653481 29570 raft_consensus.cc:348] T 00000000000000000000000000000000 P 350851ecf2bf4219bc7578b7de80ffc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "350851ecf2bf4219bc7578b7de80ffc1" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 41669 } }
I20260925 02:45:57.653797 29570 raft_consensus.cc:374] T 00000000000000000000000000000000 P 350851ecf2bf4219bc7578b7de80ffc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:45:57.653856 29570 raft_consensus.cc:729] T 00000000000000000000000000000000 P 350851ecf2bf4219bc7578b7de80ffc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 350851ecf2bf4219bc7578b7de80ffc1, State: Initialized, Role: FOLLOWER
I20260925 02:45:57.654313 29570 consensus_queue.cc:260] T 00000000000000000000000000000000 P 350851ecf2bf4219bc7578b7de80ffc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "350851ecf2bf4219bc7578b7de80ffc1" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 41669 } }
I20260925 02:45:57.654444 29570 raft_consensus.cc:388] T 00000000000000000000000000000000 P 350851ecf2bf4219bc7578b7de80ffc1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 02:45:57.654506 29570 raft_consensus.cc:482] T 00000000000000000000000000000000 P 350851ecf2bf4219bc7578b7de80ffc1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 02:45:57.654627 29570 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 350851ecf2bf4219bc7578b7de80ffc1 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:45:57.655630 29570 raft_consensus.cc:504] T 00000000000000000000000000000000 P 350851ecf2bf4219bc7578b7de80ffc1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "350851ecf2bf4219bc7578b7de80ffc1" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 41669 } }
I20260925 02:45:57.655990 29570 leader_election.cc:302] T 00000000000000000000000000000000 P 350851ecf2bf4219bc7578b7de80ffc1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 350851ecf2bf4219bc7578b7de80ffc1; no voters:
I20260925 02:45:57.656284 29570 leader_election.cc:288] T 00000000000000000000000000000000 P 350851ecf2bf4219bc7578b7de80ffc1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 02:45:57.656394 29575 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 350851ecf2bf4219bc7578b7de80ffc1 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:45:57.656594 29575 raft_consensus.cc:686] T 00000000000000000000000000000000 P 350851ecf2bf4219bc7578b7de80ffc1 [term 1 LEADER]: Becoming Leader. State: Replica: 350851ecf2bf4219bc7578b7de80ffc1, State: Running, Role: LEADER
I20260925 02:45:57.657032 29575 consensus_queue.cc:237] T 00000000000000000000000000000000 P 350851ecf2bf4219bc7578b7de80ffc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "350851ecf2bf4219bc7578b7de80ffc1" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 41669 } }
I20260925 02:45:57.657451 29570 sys_catalog.cc:564] T 00000000000000000000000000000000 P 350851ecf2bf4219bc7578b7de80ffc1 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:45:57.658722 29576 sys_catalog.cc:455] T 00000000000000000000000000000000 P 350851ecf2bf4219bc7578b7de80ffc1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "350851ecf2bf4219bc7578b7de80ffc1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "350851ecf2bf4219bc7578b7de80ffc1" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 41669 } } }
I20260925 02:45:57.658732 29577 sys_catalog.cc:455] T 00000000000000000000000000000000 P 350851ecf2bf4219bc7578b7de80ffc1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 350851ecf2bf4219bc7578b7de80ffc1. Latest consensus state: current_term: 1 leader_uuid: "350851ecf2bf4219bc7578b7de80ffc1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "350851ecf2bf4219bc7578b7de80ffc1" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 41669 } } }
I20260925 02:45:57.658874 29577 sys_catalog.cc:458] T 00000000000000000000000000000000 P 350851ecf2bf4219bc7578b7de80ffc1 [sys.catalog]: This master's current role is: LEADER
I20260925 02:45:57.658872 29576 sys_catalog.cc:458] T 00000000000000000000000000000000 P 350851ecf2bf4219bc7578b7de80ffc1 [sys.catalog]: This master's current role is: LEADER
I20260925 02:45:57.659314 29586 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:45:57.662704 29586 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:45:57.668594 29586 catalog_manager.cc:1347] Generated new cluster ID: 0f67716fe3564d8480b1345f47e8b83e
I20260925 02:45:57.668720 29586 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 02:45:57.681049 29586 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 02:45:57.681959 29586 catalog_manager.cc:1504] Loading token signing keys...
I20260925 02:45:57.687678 29586 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 350851ecf2bf4219bc7578b7de80ffc1: Generated new TSK 0
I20260925 02:45:57.688447 29586 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.157.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.157.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.157.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.129.keytab.
Entry for principal kudu/127.13.157.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.129.keytab.
I20260925 02:45:57.721947 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.157.129:0
--local_ip_for_outbound_sockets=127.13.157.129
--webserver_interface=127.13.157.129
--webserver_port=0
--tserver_master_addrs=127.13.157.190:41669
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.129.keytab
--principal=kudu/127.13.157.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:38239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:45:57.831013 29597 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:45:57.831269 29597 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:45:57.831315 29597 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:45:57.835304 29597 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:45:57.835362 29597 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:45:57.835429 29597 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.157.129
I20260925 02:45:57.840704 29597 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:38239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.129.keytab
--principal=kudu/127.13.157.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.157.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.157.129
--webserver_port=0
--tserver_master_addrs=127.13.157.190:41669
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29597
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.157.129
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:45:57.841790 29597 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:45:57.842844 29597 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:45:57.852617 29602 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:45:57.852607 29605 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:45:57.852602 29603 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:45:57.852658 29597 server_base.cc:1034] running on GCE node
I20260925 02:45:57.853189 29597 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:45:57.853865 29597 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:45:57.854971 29597 hybrid_clock.cc:648] HybridClock initialized: now 1790304357854933 us; error 50 us; skew 500 ppm
Sep 25 02:45:57 dist-test-slave-kznw krb5kdc[29489](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304357, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:45:57.857496 29597 init.cc:377] Logged in from keytab as kudu/127.13.157.129@KRBTEST.COM (short username kudu)
I20260925 02:45:57.858422 29597 webserver.cc:460] Webserver started at http://127.13.157.129:39511/ using document root <none> and password file <none>
I20260925 02:45:57.859025 29597 fs_manager.cc:359] Metadata directory not provided
I20260925 02:45:57.859098 29597 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:45:57.859337 29597 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:45:57.861368 29597 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-0/data/instance:
uuid: "4b0601a8350a44b9a3e871bdf175281b"
format_stamp: "Formatted at 2026-09-25 02:45:57 on dist-test-slave-kznw"
server_key: "9113657260e58d11ff87393cbb46c8ee"
server_key_iv: "fc69a696594b6b9a8cac4f13e8ba372f"
server_key_version: "encryptionkey@0"
I20260925 02:45:57.861912 29597 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-0/wal/instance:
uuid: "4b0601a8350a44b9a3e871bdf175281b"
format_stamp: "Formatted at 2026-09-25 02:45:57 on dist-test-slave-kznw"
server_key: "9113657260e58d11ff87393cbb46c8ee"
server_key_iv: "fc69a696594b6b9a8cac4f13e8ba372f"
server_key_version: "encryptionkey@0"
I20260925 02:45:57.865672 29597 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260925 02:45:57.868175 29612 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:45:57.869386 29597 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260925 02:45:57.869494 29597 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-0/wal
uuid: "4b0601a8350a44b9a3e871bdf175281b"
format_stamp: "Formatted at 2026-09-25 02:45:57 on dist-test-slave-kznw"
server_key: "9113657260e58d11ff87393cbb46c8ee"
server_key_iv: "fc69a696594b6b9a8cac4f13e8ba372f"
server_key_version: "encryptionkey@0"
I20260925 02:45:57.869591 29597 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:45:57.922676 29597 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:45:57.925796 29597 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:45:57.925987 29597 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:45:57.926605 29597 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:45:57.927490 29597 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:45:57.927541 29597 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:45:57.927594 29597 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:45:57.927623 29597 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:45:57.936319 29597 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.129:44449
I20260925 02:45:57.936358 29727 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.129:44449 every 8 connection(s)
I20260925 02:45:57.937240 29597 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-0/data/info.pb
I20260925 02:45:57.938892 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 29597
I20260925 02:45:57.939033 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-0/wal/instance
I20260925 02:45:57.939391 13942 external_mini_cluster.cc:1428] Setting key bb394f584acfa73bd5ad1316916ce2c4
Sep 25 02:45:57 dist-test-slave-kznw krb5kdc[29489](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304357, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.129@KRBTEST.COM for kudu/127.13.157.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.157.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.157.130@KRBTEST.COM" created.
I20260925 02:45:57.951357 29573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:57.939062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.129:38167 (local address 127.13.157.190:41669)
0925 02:45:57.939249 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0925 02:45:57.939254 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:45:57.940136 (+ 882us) server_negotiation.cc:407] Connection header received
0925 02:45:57.941665 (+ 1529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:45:57.941667 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:45:57.941713 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:45:57.941769 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:45:57.943286 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:57.943817 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:57.945014 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:57.945249 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:57.945338 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:45:57.947948 (+ 2610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:45:57.947952 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:45:57.947954 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:45:57.947980 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:45:57.949307 (+ 1327us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:45:57.949792 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:45:57.949794 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:45:57.949795 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:45:57.949867 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:45:57.950206 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:45:57.950209 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:45:57.950210 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:45:57.950401 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:45:57.950479 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0925 02:45:57.951042 (+ 563us) server_negotiation.cc:299] Negotiation successful
0925 02:45:57.951185 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260925 02:45:57.952172 29730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:57.939459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:41669 (local address 127.13.157.129:38167)
0925 02:45:57.939965 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:57.940022 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0925 02:45:57.941430 (+ 1408us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:45:57.941976 (+ 546us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:45:57.941986 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:45:57.942460 (+ 474us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:45:57.943125 (+ 665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:45:57.943141 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:45:57.943949 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:45:57.943953 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:45:57.944900 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:45:57.944911 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:45:57.945394 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:45:57.945398 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:45:57.945571 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:45:57.946117 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:45:57.946134 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:45:57.947756 (+ 1622us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:45:57.949409 (+ 1653us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:45:57.949414 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:45:57.949427 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:45:57.949705 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:45:57.949928 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:45:57.949930 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:45:57.949931 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:45:57.950084 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:45:57.950486 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:45:57.950491 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:45:57.950776 (+ 285us) client_negotiation.cc:769] Sending connection context
0925 02:45:57.950972 (+ 196us) client_negotiation.cc:240] Negotiation successful
0925 02:45:57.951313 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":120,"threads_started":1}
I20260925 02:45:57.953425 29728 heartbeater.cc:347] Connected to a master server at 127.13.157.190:41669
I20260925 02:45:57.953712 29728 heartbeater.cc:464] Registering TS with master...
I20260925 02:45:57.954224 29728 heartbeater.cc:511] Master 127.13.157.190:41669 requested a full tablet report, sending...
I20260925 02:45:57.955842 29534 ts_manager.cc:194] Registered new tserver with Master: 4b0601a8350a44b9a3e871bdf175281b (127.13.157.129:44449)
I20260925 02:45:57.957121 29534 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.157.129@KRBTEST.COM'} at 127.13.157.129:38167
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.157.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.130.keytab.
Entry for principal kudu/127.13.157.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.130.keytab.
I20260925 02:45:57.970187 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.157.130:0
--local_ip_for_outbound_sockets=127.13.157.130
--webserver_interface=127.13.157.130
--webserver_port=0
--tserver_master_addrs=127.13.157.190:41669
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.130.keytab
--principal=kudu/127.13.157.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:38239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:45:58.081256 29733 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:45:58.081521 29733 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:45:58.081563 29733 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:45:58.085498 29733 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:45:58.085556 29733 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:45:58.085616 29733 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.157.130
I20260925 02:45:58.090777 29733 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:38239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.130.keytab
--principal=kudu/127.13.157.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.157.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.157.130
--webserver_port=0
--tserver_master_addrs=127.13.157.190:41669
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29733
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.157.130
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:45:58.091832 29733 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:45:58.092823 29733 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:45:58.102433 29741 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:45:58.102442 29738 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:45:58.102595 29733 server_base.cc:1034] running on GCE node
W20260925 02:45:58.102465 29739 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:45:58.103204 29733 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:45:58.103847 29733 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:45:58.105031 29733 hybrid_clock.cc:648] HybridClock initialized: now 1790304358105013 us; error 32 us; skew 500 ppm
Sep 25 02:45:58 dist-test-slave-kznw krb5kdc[29489](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304358, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:45:58.107761 29733 init.cc:377] Logged in from keytab as kudu/127.13.157.130@KRBTEST.COM (short username kudu)
I20260925 02:45:58.108640 29733 webserver.cc:460] Webserver started at http://127.13.157.130:44459/ using document root <none> and password file <none>
I20260925 02:45:58.109141 29733 fs_manager.cc:359] Metadata directory not provided
I20260925 02:45:58.109196 29733 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:45:58.109367 29733 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:45:58.111449 29733 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-1/data/instance:
uuid: "0fb2fcd1a5744120ae6146cbc623af16"
format_stamp: "Formatted at 2026-09-25 02:45:58 on dist-test-slave-kznw"
server_key: "713b0d05899d454c583c3407fdc60fcc"
server_key_iv: "c85816a5865605371235f163aca80e25"
server_key_version: "encryptionkey@0"
I20260925 02:45:58.111981 29733 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-1/wal/instance:
uuid: "0fb2fcd1a5744120ae6146cbc623af16"
format_stamp: "Formatted at 2026-09-25 02:45:58 on dist-test-slave-kznw"
server_key: "713b0d05899d454c583c3407fdc60fcc"
server_key_iv: "c85816a5865605371235f163aca80e25"
server_key_version: "encryptionkey@0"
I20260925 02:45:58.115576 29733 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260925 02:45:58.118012 29748 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:45:58.119227 29733 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260925 02:45:58.119342 29733 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-1/wal
uuid: "0fb2fcd1a5744120ae6146cbc623af16"
format_stamp: "Formatted at 2026-09-25 02:45:58 on dist-test-slave-kznw"
server_key: "713b0d05899d454c583c3407fdc60fcc"
server_key_iv: "c85816a5865605371235f163aca80e25"
server_key_version: "encryptionkey@0"
I20260925 02:45:58.119433 29733 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:45:58.134204 29733 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:45:58.137135 29733 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:45:58.137300 29733 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:45:58.137877 29733 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:45:58.138664 29733 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:45:58.138710 29733 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:45:58.138761 29733 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:45:58.138787 29733 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:45:58.147221 29733 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.130:45427
I20260925 02:45:58.147269 29862 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.130:45427 every 8 connection(s)
I20260925 02:45:58.148105 29733 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-1/data/info.pb
I20260925 02:45:58.156519 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 29733
I20260925 02:45:58.156666 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-1/wal/instance
I20260925 02:45:58.156958 13942 external_mini_cluster.cc:1428] Setting key 5b11272fa3b76f6672161e2dd7ec25e6
Sep 25 02:45:58 dist-test-slave-kznw krb5kdc[29489](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304358, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.130@KRBTEST.COM for kudu/127.13.157.190@KRBTEST.COM
I20260925 02:45:58.161407 29573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:58.149868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.130:54027 (local address 127.13.157.190:41669)
0925 02:45:58.149993 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0925 02:45:58.149997 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:45:58.150853 (+ 856us) server_negotiation.cc:407] Connection header received
0925 02:45:58.152153 (+ 1300us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:45:58.152156 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:45:58.152212 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:45:58.152287 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:45:58.153425 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:58.153947 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:58.155084 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:58.155309 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:58.155406 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:45:58.157701 (+ 2295us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:45:58.157705 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:45:58.157707 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:45:58.157735 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:45:58.159446 (+ 1711us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:45:58.159927 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:45:58.159930 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:45:58.159932 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:45:58.160013 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:45:58.160316 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:45:58.160319 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:45:58.160321 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:45:58.160497 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:45:58.160581 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0925 02:45:58.161065 (+ 484us) server_negotiation.cc:299] Negotiation successful
0925 02:45:58.161203 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260925 02:45:58.162257 29865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:58.150276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:41669 (local address 127.13.157.130:54027)
0925 02:45:58.150708 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:58.150749 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0925 02:45:58.151940 (+ 1191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:45:58.152466 (+ 526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:45:58.152473 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:45:58.152823 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:45:58.153277 (+ 454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:45:58.153291 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:45:58.154062 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:45:58.154065 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:45:58.154969 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:45:58.154979 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:45:58.155455 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:45:58.155458 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:45:58.155611 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:45:58.156146 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:45:58.156161 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:45:58.157543 (+ 1382us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:45:58.159564 (+ 2021us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:45:58.159570 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:45:58.159584 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:45:58.159836 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:45:58.160091 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:45:58.160094 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:45:58.160096 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:45:58.160229 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:45:58.160591 (+ 362us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:45:58.160598 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:45:58.160842 (+ 244us) client_negotiation.cc:769] Sending connection context
0925 02:45:58.161008 (+ 166us) client_negotiation.cc:240] Negotiation successful
0925 02:45:58.161345 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":100,"threads_started":1}
I20260925 02:45:58.163365 29863 heartbeater.cc:347] Connected to a master server at 127.13.157.190:41669
I20260925 02:45:58.163632 29863 heartbeater.cc:464] Registering TS with master...
I20260925 02:45:58.164305 29863 heartbeater.cc:511] Master 127.13.157.190:41669 requested a full tablet report, sending...
I20260925 02:45:58.165558 29534 ts_manager.cc:194] Registered new tserver with Master: 0fb2fcd1a5744120ae6146cbc623af16 (127.13.157.130:45427)
WARNING: no policy specified for kudu/127.13.157.131@KRBTEST.COM; defaulting to no policy
I20260925 02:45:58.166335 29534 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.157.130@KRBTEST.COM'} at 127.13.157.130:54027
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.157.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.157.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.131.keytab.
Entry for principal kudu/127.13.157.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.131.keytab.
I20260925 02:45:58.186172 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.157.131:0
--local_ip_for_outbound_sockets=127.13.157.131
--webserver_interface=127.13.157.131
--webserver_port=0
--tserver_master_addrs=127.13.157.190:41669
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.131.keytab
--principal=kudu/127.13.157.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:38239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:45:58.293437 29868 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:45:58.293684 29868 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:45:58.293726 29868 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:45:58.297659 29868 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:45:58.297725 29868 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:45:58.297781 29868 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.157.131
I20260925 02:45:58.303022 29868 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:38239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.131.keytab
--principal=kudu/127.13.157.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.157.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.157.131
--webserver_port=0
--tserver_master_addrs=127.13.157.190:41669
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29868
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.157.131
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:45:58.304010 29868 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:45:58.304958 29868 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:45:58.314536 29873 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:45:58.314527 29874 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:45:58.314584 29868 server_base.cc:1034] running on GCE node
W20260925 02:45:58.314527 29876 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:45:58.315383 29868 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:45:58.316030 29868 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:45:58.317273 29868 hybrid_clock.cc:648] HybridClock initialized: now 1790304358317258 us; error 28 us; skew 500 ppm
Sep 25 02:45:58 dist-test-slave-kznw krb5kdc[29489](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304358, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:45:58.320037 29868 init.cc:377] Logged in from keytab as kudu/127.13.157.131@KRBTEST.COM (short username kudu)
I20260925 02:45:58.320875 29868 webserver.cc:460] Webserver started at http://127.13.157.131:36675/ using document root <none> and password file <none>
I20260925 02:45:58.321381 29868 fs_manager.cc:359] Metadata directory not provided
I20260925 02:45:58.321437 29868 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:45:58.321666 29868 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:45:58.323767 29868 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-2/data/instance:
uuid: "519a5d55415b4a5aa62e9ada1250a9ec"
format_stamp: "Formatted at 2026-09-25 02:45:58 on dist-test-slave-kznw"
server_key: "0f842bb50e596776dba7eab124e59a30"
server_key_iv: "adeda174ae5b415696e7759553aad175"
server_key_version: "encryptionkey@0"
I20260925 02:45:58.324245 29868 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-2/wal/instance:
uuid: "519a5d55415b4a5aa62e9ada1250a9ec"
format_stamp: "Formatted at 2026-09-25 02:45:58 on dist-test-slave-kznw"
server_key: "0f842bb50e596776dba7eab124e59a30"
server_key_iv: "adeda174ae5b415696e7759553aad175"
server_key_version: "encryptionkey@0"
I20260925 02:45:58.327806 29868 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260925 02:45:58.330098 29883 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:45:58.331396 29868 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260925 02:45:58.331496 29868 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-2/wal
uuid: "519a5d55415b4a5aa62e9ada1250a9ec"
format_stamp: "Formatted at 2026-09-25 02:45:58 on dist-test-slave-kznw"
server_key: "0f842bb50e596776dba7eab124e59a30"
server_key_iv: "adeda174ae5b415696e7759553aad175"
server_key_version: "encryptionkey@0"
I20260925 02:45:58.331601 29868 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:45:58.351853 29868 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:45:58.355044 29868 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:45:58.355224 29868 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:45:58.355845 29868 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:45:58.356673 29868 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:45:58.356719 29868 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:45:58.356761 29868 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:45:58.356789 29868 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:45:58.365198 29868 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.131:44627
I20260925 02:45:58.365227 29998 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.131:44627 every 8 connection(s)
I20260925 02:45:58.366061 29868 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-2/data/info.pb
I20260925 02:45:58.372346 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 29868
I20260925 02:45:58.372483 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-2/wal/instance
I20260925 02:45:58.372897 13942 external_mini_cluster.cc:1428] Setting key 25ae019f24734d5cf18dc09b0ecfb01a
Sep 25 02:45:58 dist-test-slave-kznw krb5kdc[29489](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304358, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.131@KRBTEST.COM for kudu/127.13.157.190@KRBTEST.COM
I20260925 02:45:58.379410 29573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:58.367862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.131:57485 (local address 127.13.157.190:41669)
0925 02:45:58.367992 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0925 02:45:58.367996 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:45:58.368831 (+ 835us) server_negotiation.cc:407] Connection header received
0925 02:45:58.370026 (+ 1195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:45:58.370029 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:45:58.370079 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:45:58.370154 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:45:58.371392 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:58.371953 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:58.373465 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:58.373707 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:58.373798 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:45:58.376285 (+ 2487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:45:58.376289 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:45:58.376291 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:45:58.376317 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:45:58.377438 (+ 1121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:45:58.377891 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:45:58.377895 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:45:58.377897 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:45:58.377974 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:45:58.378284 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:45:58.378287 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:45:58.378289 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:45:58.378468 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:45:58.378565 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0925 02:45:58.379116 (+ 551us) server_negotiation.cc:299] Negotiation successful
0925 02:45:58.379235 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260925 02:45:58.380158 30001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:58.368214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:41669 (local address 127.13.157.131:57485)
0925 02:45:58.368679 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:58.368723 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0925 02:45:58.369828 (+ 1105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:45:58.370322 (+ 494us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:45:58.370329 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:45:58.370675 (+ 346us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:45:58.371235 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:45:58.371252 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:45:58.372094 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:45:58.372099 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:45:58.373346 (+ 1247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:45:58.373356 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:45:58.373848 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:45:58.373852 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:45:58.374025 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:45:58.374636 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:45:58.374652 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:45:58.376134 (+ 1482us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:45:58.377542 (+ 1408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:45:58.377548 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:45:58.377562 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:45:58.377800 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:45:58.378054 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:45:58.378057 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:45:58.378059 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:45:58.378198 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:45:58.378566 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:45:58.378572 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:45:58.378853 (+ 281us) client_negotiation.cc:769] Sending connection context
0925 02:45:58.379047 (+ 194us) client_negotiation.cc:240] Negotiation successful
0925 02:45:58.379375 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":101,"threads_started":1}
I20260925 02:45:58.381181 29999 heartbeater.cc:347] Connected to a master server at 127.13.157.190:41669
I20260925 02:45:58.381377 29999 heartbeater.cc:464] Registering TS with master...
I20260925 02:45:58.381793 29999 heartbeater.cc:511] Master 127.13.157.190:41669 requested a full tablet report, sending...
I20260925 02:45:58.382692 29534 ts_manager.cc:194] Registered new tserver with Master: 519a5d55415b4a5aa62e9ada1250a9ec (127.13.157.131:44627)
I20260925 02:45:58.383265 29534 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.157.131@KRBTEST.COM'} at 127.13.157.131:57485
I20260925 02:45:58.386883 13942 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 02:45:58.396095 29573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:58.388861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39680 (local address 127.13.157.190:41669)
0925 02:45:58.388997 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0925 02:45:58.389001 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:45:58.389162 (+ 161us) server_negotiation.cc:407] Connection header received
0925 02:45:58.389282 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:45:58.389286 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:45:58.389330 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:45:58.389392 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:45:58.390126 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:58.390636 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:58.391521 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:58.391734 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:58.391832 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:45:58.393114 (+ 1282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:45:58.393117 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:45:58.393119 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:45:58.393142 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:45:58.394387 (+ 1245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:45:58.394791 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:45:58.394794 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:45:58.394795 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:45:58.394871 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:45:58.395177 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:45:58.395179 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:45:58.395180 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:45:58.395384 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:45:58.395487 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0925 02:45:58.395803 (+ 316us) server_negotiation.cc:299] Negotiation successful
0925 02:45:58.395941 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260925 02:45:58.398998 29534 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39680:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 02:45:58.401983 29534 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 02:45:58.415846 30013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:58.410235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.130:45427 (local address 127.0.0.1:35674)
0925 02:45:58.411121 (+ 886us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:58.411162 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0925 02:45:58.411290 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:45:58.411954 (+ 664us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:45:58.411960 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:45:58.411986 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:45:58.412129 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:45:58.412136 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:45:58.413446 (+ 1310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:45:58.413449 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:45:58.414604 (+ 1155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:45:58.414614 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:45:58.415449 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:45:58.415452 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:45:58.415528 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:45:58.415569 (+ 41us) client_negotiation.cc:769] Sending connection context
0925 02:45:58.415642 (+ 73us) client_negotiation.cc:240] Negotiation successful
0925 02:45:58.415729 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":706,"mutex_wait_us":1,"thread_start_us":93,"threads_started":1}
I20260925 02:45:58.415902 30012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:58.410371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35674 (local address 127.13.157.130:45427)
0925 02:45:58.411436 (+ 1065us) server_negotiation.cc:206] Beginning negotiation
0925 02:45:58.411447 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 02:45:58.411472 (+ 25us) server_negotiation.cc:407] Connection header received
0925 02:45:58.411551 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:45:58.411556 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:45:58.411771 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:45:58.412288 (+ 517us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:45:58.412458 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:58.413320 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:58.414720 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:58.415348 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:58.415475 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:45:58.415612 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0925 02:45:58.415667 (+ 55us) server_negotiation.cc:299] Negotiation successful
0925 02:45:58.415751 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":958,"thread_start_us":126,"threads_started":1}
I20260925 02:45:58.416904 30010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:58.410646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.129:44449 (local address 127.0.0.1:43784)
0925 02:45:58.411782 (+ 1136us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:58.411795 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 02:45:58.411869 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:45:58.412564 (+ 695us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:45:58.412567 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:45:58.412584 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:45:58.412679 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:45:58.412685 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:45:58.414088 (+ 1403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:45:58.414091 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:45:58.415279 (+ 1188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:45:58.415290 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:45:58.416481 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:45:58.416484 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:45:58.416584 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:45:58.416607 (+ 23us) client_negotiation.cc:769] Sending connection context
0925 02:45:58.416682 (+ 75us) client_negotiation.cc:240] Negotiation successful
0925 02:45:58.416768 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1081,"mutex_wait_us":25,"thread_start_us":101,"threads_started":1}
I20260925 02:45:58.417098 30014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:58.411128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43784 (local address 127.13.157.129:44449)
0925 02:45:58.411989 (+ 861us) server_negotiation.cc:206] Beginning negotiation
0925 02:45:58.412004 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0925 02:45:58.412025 (+ 21us) server_negotiation.cc:407] Connection header received
0925 02:45:58.412093 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:45:58.412099 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:45:58.412291 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:45:58.412419 (+ 128us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:45:58.412826 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:58.413950 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:58.415406 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:58.416186 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:58.416297 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:45:58.416417 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0925 02:45:58.416734 (+ 317us) server_negotiation.cc:299] Negotiation successful
0925 02:45:58.416834 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":764,"thread_start_us":113,"threads_started":1}
I20260925 02:45:58.418141 30011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:58.410387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.131:44627 (local address 127.0.0.1:51196)
0925 02:45:58.411552 (+ 1165us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:58.411563 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 02:45:58.411636 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:45:58.412284 (+ 648us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:45:58.412287 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:45:58.412306 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:45:58.412417 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:45:58.412425 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:45:58.414003 (+ 1578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:45:58.414007 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:45:58.415279 (+ 1272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:45:58.415290 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:45:58.417777 (+ 2487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:45:58.417781 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:45:58.417867 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:45:58.417888 (+ 21us) client_negotiation.cc:769] Sending connection context
0925 02:45:58.417952 (+ 64us) client_negotiation.cc:240] Negotiation successful
0925 02:45:58.418020 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1092,"mutex_wait_us":1,"thread_start_us":100,"threads_started":1}
I20260925 02:45:58.418184 30015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:58.411511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51196 (local address 127.13.157.131:44627)
0925 02:45:58.411894 (+ 383us) server_negotiation.cc:206] Beginning negotiation
0925 02:45:58.411907 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0925 02:45:58.411931 (+ 24us) server_negotiation.cc:407] Connection header received
0925 02:45:58.412004 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:45:58.412008 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:45:58.412158 (+ 150us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:45:58.412295 (+ 137us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:45:58.412533 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:58.413835 (+ 1302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:58.416141 (+ 2306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:58.417661 (+ 1520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:58.417774 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:45:58.417892 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0925 02:45:58.417978 (+ 86us) server_negotiation.cc:299] Negotiation successful
0925 02:45:58.418062 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":154,"threads_started":1}
I20260925 02:45:58.419214 29796 tablet_service.cc:1467] Processing CreateTablet for tablet 70c6a7b0906a4adf8f81e29419b039d5 (DEFAULT_TABLE table=test-table [id=eee0beafd03a4983b7bad1bdefad514f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:45:58.419864 29661 tablet_service.cc:1467] Processing CreateTablet for tablet 70c6a7b0906a4adf8f81e29419b039d5 (DEFAULT_TABLE table=test-table [id=eee0beafd03a4983b7bad1bdefad514f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:45:58.420397 29796 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 70c6a7b0906a4adf8f81e29419b039d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:45:58.420495 29931 tablet_service.cc:1467] Processing CreateTablet for tablet 70c6a7b0906a4adf8f81e29419b039d5 (DEFAULT_TABLE table=test-table [id=eee0beafd03a4983b7bad1bdefad514f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:45:58.420856 29661 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 70c6a7b0906a4adf8f81e29419b039d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:45:58.421326 29931 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 70c6a7b0906a4adf8f81e29419b039d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:45:58.426723 30016 tablet_bootstrap.cc:492] T 70c6a7b0906a4adf8f81e29419b039d5 P 519a5d55415b4a5aa62e9ada1250a9ec: Bootstrap starting.
I20260925 02:45:58.428294 30017 tablet_bootstrap.cc:492] T 70c6a7b0906a4adf8f81e29419b039d5 P 4b0601a8350a44b9a3e871bdf175281b: Bootstrap starting.
I20260925 02:45:58.428462 30018 tablet_bootstrap.cc:492] T 70c6a7b0906a4adf8f81e29419b039d5 P 0fb2fcd1a5744120ae6146cbc623af16: Bootstrap starting.
I20260925 02:45:58.428959 30016 tablet_bootstrap.cc:654] T 70c6a7b0906a4adf8f81e29419b039d5 P 519a5d55415b4a5aa62e9ada1250a9ec: Neither blocks nor log segments found. Creating new log.
I20260925 02:45:58.430007 30016 log.cc:824] T 70c6a7b0906a4adf8f81e29419b039d5 P 519a5d55415b4a5aa62e9ada1250a9ec: Log is configured to *not* fsync() on all Append() calls
I20260925 02:45:58.430449 30017 tablet_bootstrap.cc:654] T 70c6a7b0906a4adf8f81e29419b039d5 P 4b0601a8350a44b9a3e871bdf175281b: Neither blocks nor log segments found. Creating new log.
I20260925 02:45:58.430847 30018 tablet_bootstrap.cc:654] T 70c6a7b0906a4adf8f81e29419b039d5 P 0fb2fcd1a5744120ae6146cbc623af16: Neither blocks nor log segments found. Creating new log.
I20260925 02:45:58.431640 30018 log.cc:824] T 70c6a7b0906a4adf8f81e29419b039d5 P 0fb2fcd1a5744120ae6146cbc623af16: Log is configured to *not* fsync() on all Append() calls
I20260925 02:45:58.431640 30017 log.cc:824] T 70c6a7b0906a4adf8f81e29419b039d5 P 4b0601a8350a44b9a3e871bdf175281b: Log is configured to *not* fsync() on all Append() calls
I20260925 02:45:58.431761 30016 tablet_bootstrap.cc:492] T 70c6a7b0906a4adf8f81e29419b039d5 P 519a5d55415b4a5aa62e9ada1250a9ec: No bootstrap required, opened a new log
I20260925 02:45:58.431937 30016 ts_tablet_manager.cc:1397] T 70c6a7b0906a4adf8f81e29419b039d5 P 519a5d55415b4a5aa62e9ada1250a9ec: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260925 02:45:58.433590 30017 tablet_bootstrap.cc:492] T 70c6a7b0906a4adf8f81e29419b039d5 P 4b0601a8350a44b9a3e871bdf175281b: No bootstrap required, opened a new log
I20260925 02:45:58.433784 30017 ts_tablet_manager.cc:1397] T 70c6a7b0906a4adf8f81e29419b039d5 P 4b0601a8350a44b9a3e871bdf175281b: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260925 02:45:58.435607 30016 raft_consensus.cc:348] T 70c6a7b0906a4adf8f81e29419b039d5 P 519a5d55415b4a5aa62e9ada1250a9ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fb2fcd1a5744120ae6146cbc623af16" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 45427 } } peers { permanent_uuid: "519a5d55415b4a5aa62e9ada1250a9ec" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 44627 } } peers { permanent_uuid: "4b0601a8350a44b9a3e871bdf175281b" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 44449 } }
I20260925 02:45:58.435892 30016 raft_consensus.cc:374] T 70c6a7b0906a4adf8f81e29419b039d5 P 519a5d55415b4a5aa62e9ada1250a9ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:45:58.435977 30016 raft_consensus.cc:729] T 70c6a7b0906a4adf8f81e29419b039d5 P 519a5d55415b4a5aa62e9ada1250a9ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 519a5d55415b4a5aa62e9ada1250a9ec, State: Initialized, Role: FOLLOWER
I20260925 02:45:58.436026 30018 tablet_bootstrap.cc:492] T 70c6a7b0906a4adf8f81e29419b039d5 P 0fb2fcd1a5744120ae6146cbc623af16: No bootstrap required, opened a new log
I20260925 02:45:58.436200 30018 ts_tablet_manager.cc:1397] T 70c6a7b0906a4adf8f81e29419b039d5 P 0fb2fcd1a5744120ae6146cbc623af16: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.004s
I20260925 02:45:58.436439 30016 consensus_queue.cc:260] T 70c6a7b0906a4adf8f81e29419b039d5 P 519a5d55415b4a5aa62e9ada1250a9ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fb2fcd1a5744120ae6146cbc623af16" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 45427 } } peers { permanent_uuid: "519a5d55415b4a5aa62e9ada1250a9ec" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 44627 } } peers { permanent_uuid: "4b0601a8350a44b9a3e871bdf175281b" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 44449 } }
I20260925 02:45:58.437095 29999 heartbeater.cc:503] Master 127.13.157.190:41669 was elected leader, sending a full tablet report...
I20260925 02:45:58.437453 30016 ts_tablet_manager.cc:1428] T 70c6a7b0906a4adf8f81e29419b039d5 P 519a5d55415b4a5aa62e9ada1250a9ec: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260925 02:45:58.437459 30017 raft_consensus.cc:348] T 70c6a7b0906a4adf8f81e29419b039d5 P 4b0601a8350a44b9a3e871bdf175281b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fb2fcd1a5744120ae6146cbc623af16" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 45427 } } peers { permanent_uuid: "519a5d55415b4a5aa62e9ada1250a9ec" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 44627 } } peers { permanent_uuid: "4b0601a8350a44b9a3e871bdf175281b" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 44449 } }
I20260925 02:45:58.437693 30017 raft_consensus.cc:374] T 70c6a7b0906a4adf8f81e29419b039d5 P 4b0601a8350a44b9a3e871bdf175281b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:45:58.437742 30017 raft_consensus.cc:729] T 70c6a7b0906a4adf8f81e29419b039d5 P 4b0601a8350a44b9a3e871bdf175281b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b0601a8350a44b9a3e871bdf175281b, State: Initialized, Role: FOLLOWER
I20260925 02:45:58.438148 30017 consensus_queue.cc:260] T 70c6a7b0906a4adf8f81e29419b039d5 P 4b0601a8350a44b9a3e871bdf175281b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fb2fcd1a5744120ae6146cbc623af16" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 45427 } } peers { permanent_uuid: "519a5d55415b4a5aa62e9ada1250a9ec" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 44627 } } peers { permanent_uuid: "4b0601a8350a44b9a3e871bdf175281b" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 44449 } }
I20260925 02:45:58.438772 29728 heartbeater.cc:503] Master 127.13.157.190:41669 was elected leader, sending a full tablet report...
I20260925 02:45:58.439105 30017 ts_tablet_manager.cc:1428] T 70c6a7b0906a4adf8f81e29419b039d5 P 4b0601a8350a44b9a3e871bdf175281b: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
I20260925 02:45:58.439976 30018 raft_consensus.cc:348] T 70c6a7b0906a4adf8f81e29419b039d5 P 0fb2fcd1a5744120ae6146cbc623af16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fb2fcd1a5744120ae6146cbc623af16" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 45427 } } peers { permanent_uuid: "519a5d55415b4a5aa62e9ada1250a9ec" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 44627 } } peers { permanent_uuid: "4b0601a8350a44b9a3e871bdf175281b" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 44449 } }
I20260925 02:45:58.440250 30018 raft_consensus.cc:374] T 70c6a7b0906a4adf8f81e29419b039d5 P 0fb2fcd1a5744120ae6146cbc623af16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:45:58.440305 30018 raft_consensus.cc:729] T 70c6a7b0906a4adf8f81e29419b039d5 P 0fb2fcd1a5744120ae6146cbc623af16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fb2fcd1a5744120ae6146cbc623af16, State: Initialized, Role: FOLLOWER
I20260925 02:45:58.440698 30018 consensus_queue.cc:260] T 70c6a7b0906a4adf8f81e29419b039d5 P 0fb2fcd1a5744120ae6146cbc623af16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fb2fcd1a5744120ae6146cbc623af16" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 45427 } } peers { permanent_uuid: "519a5d55415b4a5aa62e9ada1250a9ec" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 44627 } } peers { permanent_uuid: "4b0601a8350a44b9a3e871bdf175281b" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 44449 } }
I20260925 02:45:58.441413 29863 heartbeater.cc:503] Master 127.13.157.190:41669 was elected leader, sending a full tablet report...
I20260925 02:45:58.441689 30018 ts_tablet_manager.cc:1428] T 70c6a7b0906a4adf8f81e29419b039d5 P 0fb2fcd1a5744120ae6146cbc623af16: Time spent starting tablet: real 0.005s user 0.004s sys 0.001s
W20260925 02:45:58.617197 30000 tablet.cc:2369] T 70c6a7b0906a4adf8f81e29419b039d5 P 519a5d55415b4a5aa62e9ada1250a9ec: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 02:45:58.649271 29864 tablet.cc:2369] T 70c6a7b0906a4adf8f81e29419b039d5 P 0fb2fcd1a5744120ae6146cbc623af16: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 02:45:58.669049 30024 raft_consensus.cc:482] T 70c6a7b0906a4adf8f81e29419b039d5 P 4b0601a8350a44b9a3e871bdf175281b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 02:45:58.669260 30024 raft_consensus.cc:504] T 70c6a7b0906a4adf8f81e29419b039d5 P 4b0601a8350a44b9a3e871bdf175281b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fb2fcd1a5744120ae6146cbc623af16" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 45427 } } peers { permanent_uuid: "519a5d55415b4a5aa62e9ada1250a9ec" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 44627 } } peers { permanent_uuid: "4b0601a8350a44b9a3e871bdf175281b" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 44449 } }
I20260925 02:45:58.670428 30024 leader_election.cc:288] T 70c6a7b0906a4adf8f81e29419b039d5 P 4b0601a8350a44b9a3e871bdf175281b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0fb2fcd1a5744120ae6146cbc623af16 (127.13.157.130:45427), 519a5d55415b4a5aa62e9ada1250a9ec (127.13.157.131:44627)
I20260925 02:45:58.675184 30012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:58.670765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.129:38503 (local address 127.13.157.130:45427)
0925 02:45:58.670992 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0925 02:45:58.670997 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:45:58.671203 (+ 206us) server_negotiation.cc:407] Connection header received
0925 02:45:58.671573 (+ 370us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:45:58.671576 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:45:58.671616 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:45:58.671681 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:45:58.671944 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:58.672681 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:58.674210 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:58.674589 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:58.674690 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:45:58.674738 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0925 02:45:58.674967 (+ 229us) server_negotiation.cc:299] Negotiation successful
0925 02:45:58.675051 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106}
I20260925 02:45:58.675186 30027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:58.670668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.130:45427 (local address 127.13.157.129:38503)
0925 02:45:58.671120 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:58.671134 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 02:45:58.671257 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:45:58.671725 (+ 468us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:45:58.671729 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:45:58.671749 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:45:58.671838 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:45:58.671842 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:45:58.672764 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:45:58.672767 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:45:58.674090 (+ 1323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:45:58.674102 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:45:58.674717 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:45:58.674721 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:45:58.674826 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:45:58.674851 (+ 25us) client_negotiation.cc:769] Sending connection context
0925 02:45:58.674938 (+ 87us) client_negotiation.cc:240] Negotiation successful
0925 02:45:58.675017 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":378,"mutex_wait_us":6,"thread_start_us":111,"threads_started":1}
I20260925 02:45:58.675249 30015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:58.670781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.129:47639 (local address 127.13.157.131:44627)
0925 02:45:58.670974 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0925 02:45:58.670979 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:45:58.671248 (+ 269us) server_negotiation.cc:407] Connection header received
0925 02:45:58.671347 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:45:58.671349 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:45:58.671396 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:45:58.671459 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:45:58.671756 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:58.672538 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:58.674252 (+ 1714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:58.674648 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:58.674776 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:45:58.674826 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0925 02:45:58.675033 (+ 207us) server_negotiation.cc:299] Negotiation successful
0925 02:45:58.675138 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260925 02:45:58.675343 30026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:58.670704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.131:44627 (local address 127.13.157.129:47639)
0925 02:45:58.671183 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:58.671194 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 02:45:58.671267 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:45:58.671481 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:45:58.671483 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:45:58.671498 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:45:58.671637 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:45:58.671643 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:45:58.672686 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:45:58.672689 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:45:58.674090 (+ 1401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:45:58.674102 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:45:58.674804 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:45:58.674808 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:45:58.674890 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:45:58.674936 (+ 46us) client_negotiation.cc:769] Sending connection context
0925 02:45:58.675007 (+ 71us) client_negotiation.cc:240] Negotiation successful
0925 02:45:58.675070 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":428,"mutex_wait_us":77,"thread_start_us":127,"threads_started":1}
I20260925 02:45:58.675781 29817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70c6a7b0906a4adf8f81e29419b039d5" candidate_uuid: "4b0601a8350a44b9a3e871bdf175281b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fb2fcd1a5744120ae6146cbc623af16" is_pre_election: true
I20260925 02:45:58.676088 29817 raft_consensus.cc:2454] T 70c6a7b0906a4adf8f81e29419b039d5 P 0fb2fcd1a5744120ae6146cbc623af16 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b0601a8350a44b9a3e871bdf175281b in term 0.
I20260925 02:45:58.676138 29952 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70c6a7b0906a4adf8f81e29419b039d5" candidate_uuid: "4b0601a8350a44b9a3e871bdf175281b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "519a5d55415b4a5aa62e9ada1250a9ec" is_pre_election: true
I20260925 02:45:58.676352 29952 raft_consensus.cc:2454] T 70c6a7b0906a4adf8f81e29419b039d5 P 519a5d55415b4a5aa62e9ada1250a9ec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b0601a8350a44b9a3e871bdf175281b in term 0.
I20260925 02:45:58.676522 29614 leader_election.cc:302] T 70c6a7b0906a4adf8f81e29419b039d5 P 4b0601a8350a44b9a3e871bdf175281b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0fb2fcd1a5744120ae6146cbc623af16, 4b0601a8350a44b9a3e871bdf175281b; no voters:
I20260925 02:45:58.676793 30024 raft_consensus.cc:2781] T 70c6a7b0906a4adf8f81e29419b039d5 P 4b0601a8350a44b9a3e871bdf175281b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 02:45:58.676870 30024 raft_consensus.cc:482] T 70c6a7b0906a4adf8f81e29419b039d5 P 4b0601a8350a44b9a3e871bdf175281b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 02:45:58.676920 30024 raft_consensus.cc:3037] T 70c6a7b0906a4adf8f81e29419b039d5 P 4b0601a8350a44b9a3e871bdf175281b [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:45:58.677904 30024 raft_consensus.cc:504] T 70c6a7b0906a4adf8f81e29419b039d5 P 4b0601a8350a44b9a3e871bdf175281b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fb2fcd1a5744120ae6146cbc623af16" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 45427 } } peers { permanent_uuid: "519a5d55415b4a5aa62e9ada1250a9ec" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 44627 } } peers { permanent_uuid: "4b0601a8350a44b9a3e871bdf175281b" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 44449 } }
I20260925 02:45:58.678226 30024 leader_election.cc:288] T 70c6a7b0906a4adf8f81e29419b039d5 P 4b0601a8350a44b9a3e871bdf175281b [CANDIDATE]: Term 1 election: Requested vote from peers 0fb2fcd1a5744120ae6146cbc623af16 (127.13.157.130:45427), 519a5d55415b4a5aa62e9ada1250a9ec (127.13.157.131:44627)
I20260925 02:45:58.678596 29952 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70c6a7b0906a4adf8f81e29419b039d5" candidate_uuid: "4b0601a8350a44b9a3e871bdf175281b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "519a5d55415b4a5aa62e9ada1250a9ec"
I20260925 02:45:58.678704 29952 raft_consensus.cc:3037] T 70c6a7b0906a4adf8f81e29419b039d5 P 519a5d55415b4a5aa62e9ada1250a9ec [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:45:58.679069 29817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70c6a7b0906a4adf8f81e29419b039d5" candidate_uuid: "4b0601a8350a44b9a3e871bdf175281b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fb2fcd1a5744120ae6146cbc623af16"
I20260925 02:45:58.679186 29817 raft_consensus.cc:3037] T 70c6a7b0906a4adf8f81e29419b039d5 P 0fb2fcd1a5744120ae6146cbc623af16 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:45:58.679831 29952 raft_consensus.cc:2454] T 70c6a7b0906a4adf8f81e29419b039d5 P 519a5d55415b4a5aa62e9ada1250a9ec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b0601a8350a44b9a3e871bdf175281b in term 1.
I20260925 02:45:58.680136 29613 leader_election.cc:302] T 70c6a7b0906a4adf8f81e29419b039d5 P 4b0601a8350a44b9a3e871bdf175281b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b0601a8350a44b9a3e871bdf175281b, 519a5d55415b4a5aa62e9ada1250a9ec; no voters:
I20260925 02:45:58.680315 29817 raft_consensus.cc:2454] T 70c6a7b0906a4adf8f81e29419b039d5 P 0fb2fcd1a5744120ae6146cbc623af16 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b0601a8350a44b9a3e871bdf175281b in term 1.
I20260925 02:45:58.680341 30024 raft_consensus.cc:2781] T 70c6a7b0906a4adf8f81e29419b039d5 P 4b0601a8350a44b9a3e871bdf175281b [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:45:58.680554 30024 raft_consensus.cc:686] T 70c6a7b0906a4adf8f81e29419b039d5 P 4b0601a8350a44b9a3e871bdf175281b [term 1 LEADER]: Becoming Leader. State: Replica: 4b0601a8350a44b9a3e871bdf175281b, State: Running, Role: LEADER
I20260925 02:45:58.680883 30024 consensus_queue.cc:237] T 70c6a7b0906a4adf8f81e29419b039d5 P 4b0601a8350a44b9a3e871bdf175281b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fb2fcd1a5744120ae6146cbc623af16" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 45427 } } peers { permanent_uuid: "519a5d55415b4a5aa62e9ada1250a9ec" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 44627 } } peers { permanent_uuid: "4b0601a8350a44b9a3e871bdf175281b" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 44449 } }
I20260925 02:45:58.683907 29534 catalog_manager.cc:5495] T 70c6a7b0906a4adf8f81e29419b039d5 P 4b0601a8350a44b9a3e871bdf175281b reported cstate change: term changed from 0 to 1, leader changed from <none> to 4b0601a8350a44b9a3e871bdf175281b (127.13.157.129). New cstate: current_term: 1 leader_uuid: "4b0601a8350a44b9a3e871bdf175281b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fb2fcd1a5744120ae6146cbc623af16" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 45427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "519a5d55415b4a5aa62e9ada1250a9ec" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 44627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b0601a8350a44b9a3e871bdf175281b" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 44449 } health_report { overall_health: HEALTHY } } }
W20260925 02:45:58.688438 29729 tablet.cc:2369] T 70c6a7b0906a4adf8f81e29419b039d5 P 4b0601a8350a44b9a3e871bdf175281b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 02:45:58.715246 30014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:58.711769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43786 (local address 127.13.157.129:44449)
0925 02:45:58.711899 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0925 02:45:58.711903 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:45:58.711918 (+ 15us) server_negotiation.cc:407] Connection header received
0925 02:45:58.712031 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:45:58.712034 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:45:58.712092 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:45:58.712192 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 02:45:58.712433 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:58.712947 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:58.713803 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:45:58.714015 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:45:58.714103 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:45:58.714408 (+ 305us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 02:45:58.714838 (+ 430us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 02:45:58.714970 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0925 02:45:58.715048 (+ 78us) server_negotiation.cc:299] Negotiation successful
0925 02:45:58.715110 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260925 02:45:58.721151 29817 raft_consensus.cc:1261] T 70c6a7b0906a4adf8f81e29419b039d5 P 0fb2fcd1a5744120ae6146cbc623af16 [term 1 FOLLOWER]: Refusing update from remote peer 4b0601a8350a44b9a3e871bdf175281b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:45:58.721151 29952 raft_consensus.cc:1261] T 70c6a7b0906a4adf8f81e29419b039d5 P 519a5d55415b4a5aa62e9ada1250a9ec [term 1 FOLLOWER]: Refusing update from remote peer 4b0601a8350a44b9a3e871bdf175281b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:45:58.721880 30028 consensus_queue.cc:1035] T 70c6a7b0906a4adf8f81e29419b039d5 P 4b0601a8350a44b9a3e871bdf175281b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0fb2fcd1a5744120ae6146cbc623af16" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 45427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:45:58.722083 30024 consensus_queue.cc:1035] T 70c6a7b0906a4adf8f81e29419b039d5 P 4b0601a8350a44b9a3e871bdf175281b [LEADER]: Connected to new peer: Peer: permanent_uuid: "519a5d55415b4a5aa62e9ada1250a9ec" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 44627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:45:58.732057 30033 mvcc.cc:205] Tried to move back new op lower bound from 7333086653314846720 to 7333086653159383040. Current Snapshot: MvccSnapshot[applied={T|T < 7333086653314846720}]
I20260925 02:45:58.739539 29534 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39680:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 02:45:58.739734 29534 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39680:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 02:45:58.742306 29534 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 350851ecf2bf4219bc7578b7de80ffc1: Sending DeleteTablet for 3 replicas of tablet 70c6a7b0906a4adf8f81e29419b039d5
I20260925 02:45:58.743109 29796 tablet_service.cc:1514] Processing DeleteTablet for tablet 70c6a7b0906a4adf8f81e29419b039d5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:45:58 UTC) from {username='kudu', principal='kudu/127.13.157.190@KRBTEST.COM'} at 127.0.0.1:35674
I20260925 02:45:58.743124 29931 tablet_service.cc:1514] Processing DeleteTablet for tablet 70c6a7b0906a4adf8f81e29419b039d5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:45:58 UTC) from {username='kudu', principal='kudu/127.13.157.190@KRBTEST.COM'} at 127.0.0.1:51196
I20260925 02:45:58.743196 29661 tablet_service.cc:1514] Processing DeleteTablet for tablet 70c6a7b0906a4adf8f81e29419b039d5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:45:58 UTC) from {username='kudu', principal='kudu/127.13.157.190@KRBTEST.COM'} at 127.0.0.1:43784
I20260925 02:45:58.743716 30043 tablet_replica.cc:330] T 70c6a7b0906a4adf8f81e29419b039d5 P 0fb2fcd1a5744120ae6146cbc623af16: stopping tablet replica
I20260925 02:45:58.743780 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 29597
I20260925 02:45:58.744050 30043 raft_consensus.cc:2229] T 70c6a7b0906a4adf8f81e29419b039d5 P 0fb2fcd1a5744120ae6146cbc623af16 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 02:45:58.744390 30043 raft_consensus.cc:2258] T 70c6a7b0906a4adf8f81e29419b039d5 P 0fb2fcd1a5744120ae6146cbc623af16 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:45:58.744865 30042 tablet_replica.cc:330] T 70c6a7b0906a4adf8f81e29419b039d5 P 519a5d55415b4a5aa62e9ada1250a9ec: stopping tablet replica
I20260925 02:45:58.745144 30042 raft_consensus.cc:2229] T 70c6a7b0906a4adf8f81e29419b039d5 P 519a5d55415b4a5aa62e9ada1250a9ec [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 02:45:58.745404 30042 raft_consensus.cc:2258] T 70c6a7b0906a4adf8f81e29419b039d5 P 519a5d55415b4a5aa62e9ada1250a9ec [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:45:58.746510 30043 ts_tablet_manager.cc:1905] T 70c6a7b0906a4adf8f81e29419b039d5 P 0fb2fcd1a5744120ae6146cbc623af16: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:45:58.746672 30042 ts_tablet_manager.cc:1905] T 70c6a7b0906a4adf8f81e29419b039d5 P 519a5d55415b4a5aa62e9ada1250a9ec: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:45:58.748942 30043 ts_tablet_manager.cc:1918] T 70c6a7b0906a4adf8f81e29419b039d5 P 0fb2fcd1a5744120ae6146cbc623af16: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 02:45:58.749043 30043 log.cc:1196] T 70c6a7b0906a4adf8f81e29419b039d5 P 0fb2fcd1a5744120ae6146cbc623af16: Deleting WAL directory at /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-1/wal/wals/70c6a7b0906a4adf8f81e29419b039d5
I20260925 02:45:58.749250 30042 ts_tablet_manager.cc:1918] T 70c6a7b0906a4adf8f81e29419b039d5 P 519a5d55415b4a5aa62e9ada1250a9ec: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 02:45:58.749342 30043 ts_tablet_manager.cc:1939] T 70c6a7b0906a4adf8f81e29419b039d5 P 0fb2fcd1a5744120ae6146cbc623af16: Deleting consensus metadata
I20260925 02:45:58.749348 30042 log.cc:1196] T 70c6a7b0906a4adf8f81e29419b039d5 P 519a5d55415b4a5aa62e9ada1250a9ec: Deleting WAL directory at /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-2/wal/wals/70c6a7b0906a4adf8f81e29419b039d5
I20260925 02:45:58.749646 30042 ts_tablet_manager.cc:1939] T 70c6a7b0906a4adf8f81e29419b039d5 P 519a5d55415b4a5aa62e9ada1250a9ec: Deleting consensus metadata
I20260925 02:45:58.750573 29519 catalog_manager.cc:4841] TS 0fb2fcd1a5744120ae6146cbc623af16 (127.13.157.130:45427): tablet 70c6a7b0906a4adf8f81e29419b039d5 (table test-table [id=eee0beafd03a4983b7bad1bdefad514f]) successfully deleted
I20260925 02:45:58.750689 29518 catalog_manager.cc:4841] TS 519a5d55415b4a5aa62e9ada1250a9ec (127.13.157.131:44627): tablet 70c6a7b0906a4adf8f81e29419b039d5 (table test-table [id=eee0beafd03a4983b7bad1bdefad514f]) successfully deleted
W20260925 02:45:58.751758 29520 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.157.129:44449 (error 108)
I20260925 02:45:58.751948 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 29733
I20260925 02:45:58.752843 30011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:45:58.752539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.129:44449 (local address 127.0.0.1:43800)
0925 02:45:58.752674 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0925 02:45:58.752750 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.129:44449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260925 02:45:58.752981 29520 catalog_manager.cc:4584] TS 4b0601a8350a44b9a3e871bdf175281b (127.13.157.129:44449): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 70c6a7b0906a4adf8f81e29419b039d5: Network error: Client connection negotiation failed: client connection to 127.13.157.129:44449: connect: Connection refused (error 111)
I20260925 02:45:58.757793 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 29868
I20260925 02:45:58.763283 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 29502
I20260925 02:46:18.769294 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.157.190:41669
--webserver_interface=127.13.157.190
--webserver_port=34797
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab
--principal=kudu/127.13.157.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:38239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.157.190:41669
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:46:18.876991 30045 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:46:18.877245 30045 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:46:18.877283 30045 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:46:18.881296 30045 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260925 02:46:18.881350 30045 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260925 02:46:18.881366 30045 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260925 02:46:18.881377 30045 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:46:18.881392 30045 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:46:18.881412 30045 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:46:18.881441 30045 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:46:18.881469 30045 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:46:18.886705 30045 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:38239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.13.157.190:41669
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab
--principal=kudu/127.13.157.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.157.190:41669
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.157.190
--webserver_port=34797
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30045
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:46:18.887802 30045 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:46:18.888871 30045 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:46:18.901916 30052 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:46:18.902124 30045 server_base.cc:1034] running on GCE node
W20260925 02:46:18.901927 30054 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:46:18.901935 30051 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:46:18.902834 30045 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:46:18.904064 30045 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:46:18.905225 30045 hybrid_clock.cc:648] HybridClock initialized: now 1790304378905210 us; error 41 us; skew 500 ppm
Sep 25 02:46:18 dist-test-slave-kznw krb5kdc[29489](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304378, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:46:18.911993 30045 init.cc:377] Logged in from keytab as kudu/127.13.157.190@KRBTEST.COM (short username kudu)
I20260925 02:46:18.913000 30045 webserver.cc:460] Webserver started at http://127.13.157.190:34797/ using document root <none> and password file <none>
I20260925 02:46:18.913579 30045 fs_manager.cc:359] Metadata directory not provided
I20260925 02:46:18.913667 30045 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:46:18.917850 30045 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260925 02:46:18.920270 30061 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:46:18.921593 30045 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260925 02:46:18.921713 30045 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/master-0/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/master-0/wal
uuid: "350851ecf2bf4219bc7578b7de80ffc1"
format_stamp: "Formatted at 2026-09-25 02:45:57 on dist-test-slave-kznw"
server_key: "c3fe0d9e324b624e722df0418a656b85"
server_key_iv: "f23e3496dfb85ca0bcd7e73cb0a7c965"
server_key_version: "encryptionkey@0"
I20260925 02:46:18.922293 30045 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:46:18.940092 30045 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:46:18.943300 30045 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:46:18.943459 30045 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:46:18.950357 30045 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.190:41669
I20260925 02:46:18.950625 30115 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.190:41669 every 8 connection(s)
I20260925 02:46:18.951483 30045 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
I20260925 02:46:18.955628 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 30045
I20260925 02:46:18.955996 30116 sys_catalog.cc:263] Verifying existing consensus state
I20260925 02:46:18.956037 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.157.129:44449
--local_ip_for_outbound_sockets=127.13.157.129
--tserver_master_addrs=127.13.157.190:41669
--webserver_port=39511
--webserver_interface=127.13.157.129
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.129.keytab
--principal=kudu/127.13.157.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:38239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 02:46:18.957664 30116 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 350851ecf2bf4219bc7578b7de80ffc1: Bootstrap starting.
I20260925 02:46:18.974376 30116 log.cc:824] T 00000000000000000000000000000000 P 350851ecf2bf4219bc7578b7de80ffc1: Log is configured to *not* fsync() on all Append() calls
I20260925 02:46:18.983345 30116 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 350851ecf2bf4219bc7578b7de80ffc1: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 02:46:18.983695 30116 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 350851ecf2bf4219bc7578b7de80ffc1: Bootstrap complete.
I20260925 02:46:18.987542 30116 raft_consensus.cc:348] T 00000000000000000000000000000000 P 350851ecf2bf4219bc7578b7de80ffc1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "350851ecf2bf4219bc7578b7de80ffc1" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 41669 } }
I20260925 02:46:18.988049 30116 raft_consensus.cc:729] T 00000000000000000000000000000000 P 350851ecf2bf4219bc7578b7de80ffc1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 350851ecf2bf4219bc7578b7de80ffc1, State: Initialized, Role: FOLLOWER
I20260925 02:46:18.988507 30116 consensus_queue.cc:260] T 00000000000000000000000000000000 P 350851ecf2bf4219bc7578b7de80ffc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "350851ecf2bf4219bc7578b7de80ffc1" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 41669 } }
I20260925 02:46:18.988628 30116 raft_consensus.cc:388] T 00000000000000000000000000000000 P 350851ecf2bf4219bc7578b7de80ffc1 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 02:46:18.988703 30116 raft_consensus.cc:482] T 00000000000000000000000000000000 P 350851ecf2bf4219bc7578b7de80ffc1 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 02:46:18.988798 30116 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 350851ecf2bf4219bc7578b7de80ffc1 [term 1 FOLLOWER]: Advancing to term 2
I20260925 02:46:18.990715 30116 raft_consensus.cc:504] T 00000000000000000000000000000000 P 350851ecf2bf4219bc7578b7de80ffc1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "350851ecf2bf4219bc7578b7de80ffc1" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 41669 } }
I20260925 02:46:18.991061 30116 leader_election.cc:302] T 00000000000000000000000000000000 P 350851ecf2bf4219bc7578b7de80ffc1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 350851ecf2bf4219bc7578b7de80ffc1; no voters:
I20260925 02:46:18.991360 30116 leader_election.cc:288] T 00000000000000000000000000000000 P 350851ecf2bf4219bc7578b7de80ffc1 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260925 02:46:18.991492 30120 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 350851ecf2bf4219bc7578b7de80ffc1 [term 2 FOLLOWER]: Leader election won for term 2
I20260925 02:46:18.991765 30120 raft_consensus.cc:686] T 00000000000000000000000000000000 P 350851ecf2bf4219bc7578b7de80ffc1 [term 2 LEADER]: Becoming Leader. State: Replica: 350851ecf2bf4219bc7578b7de80ffc1, State: Running, Role: LEADER
I20260925 02:46:18.992106 30120 consensus_queue.cc:237] T 00000000000000000000000000000000 P 350851ecf2bf4219bc7578b7de80ffc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "350851ecf2bf4219bc7578b7de80ffc1" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 41669 } }
I20260925 02:46:18.992471 30116 sys_catalog.cc:564] T 00000000000000000000000000000000 P 350851ecf2bf4219bc7578b7de80ffc1 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:46:18.993080 30120 sys_catalog.cc:455] T 00000000000000000000000000000000 P 350851ecf2bf4219bc7578b7de80ffc1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 350851ecf2bf4219bc7578b7de80ffc1. Latest consensus state: current_term: 2 leader_uuid: "350851ecf2bf4219bc7578b7de80ffc1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "350851ecf2bf4219bc7578b7de80ffc1" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 41669 } } }
I20260925 02:46:18.993201 30120 sys_catalog.cc:458] T 00000000000000000000000000000000 P 350851ecf2bf4219bc7578b7de80ffc1 [sys.catalog]: This master's current role is: LEADER
I20260925 02:46:18.993649 30121 sys_catalog.cc:455] T 00000000000000000000000000000000 P 350851ecf2bf4219bc7578b7de80ffc1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "350851ecf2bf4219bc7578b7de80ffc1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "350851ecf2bf4219bc7578b7de80ffc1" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 41669 } } }
I20260925 02:46:18.993758 30121 sys_catalog.cc:458] T 00000000000000000000000000000000 P 350851ecf2bf4219bc7578b7de80ffc1 [sys.catalog]: This master's current role is: LEADER
I20260925 02:46:18.997310 30134 catalog_manager.cc:1259] Loaded cluster ID: 0f67716fe3564d8480b1345f47e8b83e
I20260925 02:46:18.997357 30134 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 350851ecf2bf4219bc7578b7de80ffc1: loading cluster ID for follower catalog manager: success
I20260925 02:46:18.999143 30134 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 350851ecf2bf4219bc7578b7de80ffc1: acquiring CA information for follower catalog manager: success
I20260925 02:46:19.000392 30134 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 350851ecf2bf4219bc7578b7de80ffc1: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 02:46:19.000502 30135 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:46:19.002421 30135 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:46:19.002841 30135 catalog_manager.cc:1259] Loaded cluster ID: 0f67716fe3564d8480b1345f47e8b83e
I20260925 02:46:19.002889 30135 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 02:46:19.003432 30135 catalog_manager.cc:1504] Loading token signing keys...
I20260925 02:46:19.003827 30135 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 350851ecf2bf4219bc7578b7de80ffc1: Loaded TSK: 0
I20260925 02:46:19.015755 30135 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 350851ecf2bf4219bc7578b7de80ffc1: Generated new TSK 1
I20260925 02:46:19.015985 30135 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260925 02:46:19.079576 30118 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:46:19.079861 30118 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:46:19.079924 30118 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:46:19.084072 30118 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:46:19.084146 30118 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:46:19.084225 30118 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.157.129
I20260925 02:46:19.089815 30118 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:38239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.129.keytab
--principal=kudu/127.13.157.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.157.129:44449
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.157.129
--webserver_port=39511
--tserver_master_addrs=127.13.157.190:41669
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30118
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.157.129
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:46:19.090894 30118 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:46:19.092023 30118 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:46:19.101820 30144 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:46:19.101807 30143 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:46:19.101807 30146 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:46:19.101879 30118 server_base.cc:1034] running on GCE node
I20260925 02:46:19.102447 30118 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:46:19.103132 30118 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:46:19.104307 30118 hybrid_clock.cc:648] HybridClock initialized: now 1790304379104278 us; error 44 us; skew 500 ppm
Sep 25 02:46:19 dist-test-slave-kznw krb5kdc[29489](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304379, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:46:19.106956 30118 init.cc:377] Logged in from keytab as kudu/127.13.157.129@KRBTEST.COM (short username kudu)
I20260925 02:46:19.107864 30118 webserver.cc:460] Webserver started at http://127.13.157.129:39511/ using document root <none> and password file <none>
I20260925 02:46:19.108402 30118 fs_manager.cc:359] Metadata directory not provided
I20260925 02:46:19.108479 30118 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:46:19.112233 30118 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260925 02:46:19.114254 30153 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:46:19.115489 30118 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260925 02:46:19.115592 30118 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-0/wal
uuid: "4b0601a8350a44b9a3e871bdf175281b"
format_stamp: "Formatted at 2026-09-25 02:45:57 on dist-test-slave-kznw"
server_key: "9113657260e58d11ff87393cbb46c8ee"
server_key_iv: "fc69a696594b6b9a8cac4f13e8ba372f"
server_key_version: "encryptionkey@0"
I20260925 02:46:19.116119 30118 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:46:19.160633 30118 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:46:19.165053 30118 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:46:19.165257 30118 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:46:19.166103 30118 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:46:19.167614 30160 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260925 02:46:19.172289 30118 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260925 02:46:19.172350 30118 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20260925 02:46:19.172416 30118 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260925 02:46:19.174660 30118 ts_tablet_manager.cc:610] Registered 1 tablets
I20260925 02:46:19.174726 30118 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260925 02:46:19.175050 30160 tablet_bootstrap.cc:492] T 70c6a7b0906a4adf8f81e29419b039d5 P 4b0601a8350a44b9a3e871bdf175281b: Bootstrap starting.
I20260925 02:46:19.185449 30118 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.129:44449
I20260925 02:46:19.185691 30267 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.129:44449 every 8 connection(s)
I20260925 02:46:19.186679 30118 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-0/data/info.pb
I20260925 02:46:19.193025 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 30118
I20260925 02:46:19.193595 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.157.130:45427
--local_ip_for_outbound_sockets=127.13.157.130
--tserver_master_addrs=127.13.157.190:41669
--webserver_port=44459
--webserver_interface=127.13.157.130
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.130.keytab
--principal=kudu/127.13.157.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:38239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 02:46:19.195808 30160 log.cc:824] T 70c6a7b0906a4adf8f81e29419b039d5 P 4b0601a8350a44b9a3e871bdf175281b: Log is configured to *not* fsync() on all Append() calls
Sep 25 02:46:19 dist-test-slave-kznw krb5kdc[29489](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304379, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.129@KRBTEST.COM for kudu/127.13.157.190@KRBTEST.COM
I20260925 02:46:19.204064 30160 tablet_bootstrap.cc:492] T 70c6a7b0906a4adf8f81e29419b039d5 P 4b0601a8350a44b9a3e871bdf175281b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 02:46:19.204632 30160 tablet_bootstrap.cc:492] T 70c6a7b0906a4adf8f81e29419b039d5 P 4b0601a8350a44b9a3e871bdf175281b: Bootstrap complete.
I20260925 02:46:19.206020 30160 ts_tablet_manager.cc:1397] T 70c6a7b0906a4adf8f81e29419b039d5 P 4b0601a8350a44b9a3e871bdf175281b: Time spent bootstrapping tablet: real 0.031s user 0.018s sys 0.011s
I20260925 02:46:19.209717 30271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:19.189470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:41669 (local address 127.13.157.129:54497)
0925 02:46:19.191651 (+ 2181us) negotiation.cc:107] Waiting for socket to connect
0925 02:46:19.191712 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0925 02:46:19.193274 (+ 1562us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:46:19.195374 (+ 2100us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:46:19.195383 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:46:19.195883 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:46:19.196662 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:46:19.196684 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:46:19.198489 (+ 1805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:46:19.198492 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:46:19.199906 (+ 1414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:46:19.199918 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:46:19.201014 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:46:19.201018 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:46:19.201238 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:46:19.201898 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:46:19.201919 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:46:19.203844 (+ 1925us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:46:19.206367 (+ 2523us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:46:19.206373 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:46:19.206386 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:46:19.206713 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:46:19.207115 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:46:19.207118 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:46:19.207120 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:46:19.207301 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:46:19.207997 (+ 696us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:46:19.208003 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:46:19.208285 (+ 282us) client_negotiation.cc:769] Sending connection context
0925 02:46:19.208457 (+ 172us) client_negotiation.cc:240] Negotiation successful
0925 02:46:19.208766 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1949,"thread_start_us":165,"threads_started":1}
I20260925 02:46:19.209244 30160 raft_consensus.cc:348] T 70c6a7b0906a4adf8f81e29419b039d5 P 4b0601a8350a44b9a3e871bdf175281b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fb2fcd1a5744120ae6146cbc623af16" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 45427 } } peers { permanent_uuid: "519a5d55415b4a5aa62e9ada1250a9ec" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 44627 } } peers { permanent_uuid: "4b0601a8350a44b9a3e871bdf175281b" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 44449 } }
I20260925 02:46:19.210215 30160 raft_consensus.cc:729] T 70c6a7b0906a4adf8f81e29419b039d5 P 4b0601a8350a44b9a3e871bdf175281b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b0601a8350a44b9a3e871bdf175281b, State: Initialized, Role: FOLLOWER
I20260925 02:46:19.210877 30160 consensus_queue.cc:260] T 70c6a7b0906a4adf8f81e29419b039d5 P 4b0601a8350a44b9a3e871bdf175281b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fb2fcd1a5744120ae6146cbc623af16" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 45427 } } peers { permanent_uuid: "519a5d55415b4a5aa62e9ada1250a9ec" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 44627 } } peers { permanent_uuid: "4b0601a8350a44b9a3e871bdf175281b" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 44449 } }
I20260925 02:46:19.211462 30270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:19.189419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.129:54497 (local address 127.13.157.190:41669)
0925 02:46:19.191339 (+ 1920us) server_negotiation.cc:206] Beginning negotiation
0925 02:46:19.191360 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0925 02:46:19.191863 (+ 503us) server_negotiation.cc:407] Connection header received
0925 02:46:19.193838 (+ 1975us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:46:19.193858 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:46:19.194290 (+ 432us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:46:19.194637 (+ 347us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:46:19.196871 (+ 2234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:46:19.198296 (+ 1425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:46:19.200069 (+ 1773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:46:19.200829 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:46:19.201033 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:46:19.204082 (+ 3049us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:46:19.204089 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:46:19.204106 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:46:19.204139 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:46:19.206235 (+ 2096us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:46:19.206837 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:46:19.206843 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:46:19.206849 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:46:19.207010 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:46:19.207402 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:46:19.207406 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:46:19.207408 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:46:19.207829 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:46:19.207944 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0925 02:46:19.210301 (+ 2357us) server_negotiation.cc:299] Negotiation successful
0925 02:46:19.210597 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1740,"thread_start_us":132,"threads_started":1}
I20260925 02:46:19.212338 30160 ts_tablet_manager.cc:1428] T 70c6a7b0906a4adf8f81e29419b039d5 P 4b0601a8350a44b9a3e871bdf175281b: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260925 02:46:19.213959 30268 heartbeater.cc:347] Connected to a master server at 127.13.157.190:41669
I20260925 02:46:19.214195 30268 heartbeater.cc:464] Registering TS with master...
I20260925 02:46:19.214669 30268 heartbeater.cc:511] Master 127.13.157.190:41669 requested a full tablet report, sending...
I20260925 02:46:19.217142 30079 ts_manager.cc:194] Registered new tserver with Master: 4b0601a8350a44b9a3e871bdf175281b (127.13.157.129:44449)
I20260925 02:46:19.221239 30079 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.157.129@KRBTEST.COM'} at 127.13.157.129:54497
I20260925 02:46:19.223791 30268 heartbeater.cc:503] Master 127.13.157.190:41669 was elected leader, sending a full tablet report...
Sep 25 02:46:19 dist-test-slave-kznw krb5kdc[29489](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304378, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.190@KRBTEST.COM for kudu/127.13.157.129@KRBTEST.COM
I20260925 02:46:19.230669 30275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:19.220545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.129:44449 (local address 127.0.0.1:56970)
0925 02:46:19.221279 (+ 734us) negotiation.cc:107] Waiting for socket to connect
0925 02:46:19.221311 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0925 02:46:19.221494 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:46:19.221930 (+ 436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:46:19.221935 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:46:19.222317 (+ 382us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:46:19.222449 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:46:19.222457 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:46:19.223801 (+ 1344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:46:19.223804 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:46:19.224592 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:46:19.224600 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:46:19.225118 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:46:19.225122 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:46:19.225242 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:46:19.225808 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:46:19.225819 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:46:19.227330 (+ 1511us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:46:19.229053 (+ 1723us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:46:19.229063 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:46:19.229068 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:46:19.229286 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:46:19.229555 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:46:19.229558 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:46:19.229560 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:46:19.229639 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:46:19.230062 (+ 423us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:46:19.230068 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:46:19.230187 (+ 119us) client_negotiation.cc:769] Sending connection context
0925 02:46:19.230302 (+ 115us) client_negotiation.cc:240] Negotiation successful
0925 02:46:19.230492 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":595,"thread_start_us":138,"threads_started":1}
I20260925 02:46:19.230698 30276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:19.220641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56970 (local address 127.13.157.129:44449)
0925 02:46:19.221036 (+ 395us) server_negotiation.cc:206] Beginning negotiation
0925 02:46:19.221046 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 02:46:19.221396 (+ 350us) server_negotiation.cc:407] Connection header received
0925 02:46:19.221644 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:46:19.221648 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:46:19.221798 (+ 150us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:46:19.221912 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:46:19.222570 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:46:19.223678 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:46:19.224701 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:46:19.225014 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:46:19.225104 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:46:19.227456 (+ 2352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:46:19.227465 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:46:19.227474 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:46:19.227494 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:46:19.228950 (+ 1456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:46:19.229389 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:46:19.229395 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:46:19.229397 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:46:19.229472 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:46:19.229718 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:46:19.229721 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:46:19.229723 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:46:19.229955 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:46:19.230071 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0925 02:46:19.230396 (+ 325us) server_negotiation.cc:299] Negotiation successful
0925 02:46:19.230547 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":98,"threads_started":1}
I20260925 02:46:19.231700 30202 tablet_service.cc:1514] Processing DeleteTablet for tablet 70c6a7b0906a4adf8f81e29419b039d5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:45:58 UTC) from {username='kudu', principal='kudu/127.13.157.190@KRBTEST.COM'} at 127.0.0.1:56970
I20260925 02:46:19.232693 30277 tablet_replica.cc:330] stopping tablet replica
I20260925 02:46:19.233035 30277 raft_consensus.cc:2229] T 70c6a7b0906a4adf8f81e29419b039d5 P 4b0601a8350a44b9a3e871bdf175281b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 02:46:19.233278 30277 raft_consensus.cc:2258] T 70c6a7b0906a4adf8f81e29419b039d5 P 4b0601a8350a44b9a3e871bdf175281b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:46:19.234212 30277 ts_tablet_manager.cc:1905] T 70c6a7b0906a4adf8f81e29419b039d5 P 4b0601a8350a44b9a3e871bdf175281b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:46:19.237939 30277 ts_tablet_manager.cc:1918] T 70c6a7b0906a4adf8f81e29419b039d5 P 4b0601a8350a44b9a3e871bdf175281b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 02:46:19.238025 30277 log.cc:1196] T 70c6a7b0906a4adf8f81e29419b039d5 P 4b0601a8350a44b9a3e871bdf175281b: Deleting WAL directory at /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-0/wal/wals/70c6a7b0906a4adf8f81e29419b039d5
I20260925 02:46:19.238273 30277 ts_tablet_manager.cc:1939] T 70c6a7b0906a4adf8f81e29419b039d5 P 4b0601a8350a44b9a3e871bdf175281b: Deleting consensus metadata
I20260925 02:46:19.239900 30064 catalog_manager.cc:4841] TS 4b0601a8350a44b9a3e871bdf175281b (127.13.157.129:44449): tablet 70c6a7b0906a4adf8f81e29419b039d5 (table test-table [id=eee0beafd03a4983b7bad1bdefad514f]) successfully deleted
W20260925 02:46:19.312696 30272 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:46:19.312963 30272 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:46:19.313016 30272 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:46:19.317055 30272 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:46:19.317123 30272 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:46:19.317183 30272 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.157.130
I20260925 02:46:19.322638 30272 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:38239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.130.keytab
--principal=kudu/127.13.157.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.157.130:45427
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.157.130
--webserver_port=44459
--tserver_master_addrs=127.13.157.190:41669
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30272
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.157.130
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:46:19.323700 30272 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:46:19.324704 30272 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:46:19.333946 30282 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:46:19.334092 30272 server_base.cc:1034] running on GCE node
W20260925 02:46:19.333954 30285 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:46:19.334197 30283 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:46:19.334656 30272 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:46:19.335314 30272 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:46:19.336475 30272 hybrid_clock.cc:648] HybridClock initialized: now 1790304379336457 us; error 27 us; skew 500 ppm
Sep 25 02:46:19 dist-test-slave-kznw krb5kdc[29489](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304379, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:46:19.339090 30272 init.cc:377] Logged in from keytab as kudu/127.13.157.130@KRBTEST.COM (short username kudu)
I20260925 02:46:19.339929 30272 webserver.cc:460] Webserver started at http://127.13.157.130:44459/ using document root <none> and password file <none>
I20260925 02:46:19.340428 30272 fs_manager.cc:359] Metadata directory not provided
I20260925 02:46:19.340498 30272 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:46:19.344190 30272 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260925 02:46:19.346143 30292 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:46:19.347365 30272 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260925 02:46:19.347458 30272 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-1/wal
uuid: "0fb2fcd1a5744120ae6146cbc623af16"
format_stamp: "Formatted at 2026-09-25 02:45:58 on dist-test-slave-kznw"
server_key: "713b0d05899d454c583c3407fdc60fcc"
server_key_iv: "c85816a5865605371235f163aca80e25"
server_key_version: "encryptionkey@0"
I20260925 02:46:19.347975 30272 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:46:19.371475 30272 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:46:19.374697 30272 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:46:19.374871 30272 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:46:19.375494 30272 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:46:19.376327 30272 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:46:19.376371 30272 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:46:19.376410 30272 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:46:19.376423 30272 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:46:19.385057 30272 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.130:45427
I20260925 02:46:19.385089 30406 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.130:45427 every 8 connection(s)
I20260925 02:46:19.386027 30272 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-1/data/info.pb
I20260925 02:46:19.391476 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 30272
I20260925 02:46:19.391896 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.157.131:44627
--local_ip_for_outbound_sockets=127.13.157.131
--tserver_master_addrs=127.13.157.190:41669
--webserver_port=36675
--webserver_interface=127.13.157.131
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.131.keytab
--principal=kudu/127.13.157.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:38239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 25 02:46:19 dist-test-slave-kznw krb5kdc[29489](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304379, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.130@KRBTEST.COM for kudu/127.13.157.190@KRBTEST.COM
I20260925 02:46:19.400172 30270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:19.387808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.130:54547 (local address 127.13.157.190:41669)
0925 02:46:19.387975 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0925 02:46:19.387980 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:46:19.388831 (+ 851us) server_negotiation.cc:407] Connection header received
0925 02:46:19.390026 (+ 1195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:46:19.390029 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:46:19.390077 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:46:19.390149 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:46:19.391303 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:46:19.391868 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:46:19.393457 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:46:19.393827 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:46:19.393933 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:46:19.396645 (+ 2712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:46:19.396648 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:46:19.396650 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:46:19.396678 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:46:19.398028 (+ 1350us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:46:19.398523 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:46:19.398526 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:46:19.398527 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:46:19.398608 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:46:19.398990 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:46:19.398994 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:46:19.398997 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:46:19.399201 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:46:19.399302 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0925 02:46:19.399845 (+ 543us) server_negotiation.cc:299] Negotiation successful
0925 02:46:19.399989 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260925 02:46:19.400990 30409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:19.388252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:41669 (local address 127.13.157.130:54547)
0925 02:46:19.388684 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0925 02:46:19.388726 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0925 02:46:19.389830 (+ 1104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:46:19.390317 (+ 487us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:46:19.390324 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:46:19.390678 (+ 354us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:46:19.391180 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:46:19.391193 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:46:19.391997 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:46:19.392001 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:46:19.393312 (+ 1311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:46:19.393325 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:46:19.393971 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:46:19.393974 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:46:19.394150 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:46:19.394726 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:46:19.394743 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:46:19.396465 (+ 1722us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:46:19.398145 (+ 1680us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:46:19.398151 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:46:19.398168 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:46:19.398424 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:46:19.398717 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:46:19.398720 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:46:19.398722 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:46:19.398873 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:46:19.399314 (+ 441us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:46:19.399321 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:46:19.399593 (+ 272us) client_negotiation.cc:769] Sending connection context
0925 02:46:19.399768 (+ 175us) client_negotiation.cc:240] Negotiation successful
0925 02:46:19.400109 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":112,"threads_started":1}
I20260925 02:46:19.402312 30407 heartbeater.cc:347] Connected to a master server at 127.13.157.190:41669
I20260925 02:46:19.402536 30407 heartbeater.cc:464] Registering TS with master...
I20260925 02:46:19.403034 30407 heartbeater.cc:511] Master 127.13.157.190:41669 requested a full tablet report, sending...
I20260925 02:46:19.404131 30079 ts_manager.cc:194] Registered new tserver with Master: 0fb2fcd1a5744120ae6146cbc623af16 (127.13.157.130:45427)
I20260925 02:46:19.404891 30079 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.157.130@KRBTEST.COM'} at 127.13.157.130:54547
W20260925 02:46:19.502449 30410 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:46:19.502678 30410 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:46:19.502719 30410 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:46:19.506621 30410 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:46:19.506675 30410 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:46:19.506729 30410 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.157.131
I20260925 02:46:19.511793 30410 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:38239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.131.keytab
--principal=kudu/127.13.157.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.157.131:44627
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.157.131
--webserver_port=36675
--tserver_master_addrs=127.13.157.190:41669
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30410
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.157.131
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:46:19.512729 30410 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:46:19.513648 30410 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:46:19.522886 30418 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:46:19.522850 30415 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:46:19.522929 30416 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:46:19.523044 30410 server_base.cc:1034] running on GCE node
I20260925 02:46:19.523519 30410 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:46:19.524113 30410 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:46:19.525276 30410 hybrid_clock.cc:648] HybridClock initialized: now 1790304379525257 us; error 31 us; skew 500 ppm
Sep 25 02:46:19 dist-test-slave-kznw krb5kdc[29489](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304379, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:46:19.527940 30410 init.cc:377] Logged in from keytab as kudu/127.13.157.131@KRBTEST.COM (short username kudu)
I20260925 02:46:19.528816 30410 webserver.cc:460] Webserver started at http://127.13.157.131:36675/ using document root <none> and password file <none>
I20260925 02:46:19.529318 30410 fs_manager.cc:359] Metadata directory not provided
I20260925 02:46:19.529381 30410 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:46:19.533102 30410 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260925 02:46:19.535032 30425 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:46:19.536198 30410 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260925 02:46:19.536307 30410 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-2/wal
uuid: "519a5d55415b4a5aa62e9ada1250a9ec"
format_stamp: "Formatted at 2026-09-25 02:45:58 on dist-test-slave-kznw"
server_key: "0f842bb50e596776dba7eab124e59a30"
server_key_iv: "adeda174ae5b415696e7759553aad175"
server_key_version: "encryptionkey@0"
I20260925 02:46:19.536799 30410 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:46:19.554198 30410 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:46:19.556901 30410 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:46:19.557070 30410 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:46:19.557644 30410 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:46:19.558462 30410 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:46:19.558508 30410 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:46:19.558560 30410 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:46:19.558585 30410 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:46:19.566679 30410 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.131:44627
I20260925 02:46:19.566732 30540 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.131:44627 every 8 connection(s)
I20260925 02:46:19.567749 30410 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-2/data/info.pb
I20260925 02:46:19.568746 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 30410
I20260925 02:46:19.577751 30270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:19.569454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44898 (local address 127.13.157.190:41669)
0925 02:46:19.569573 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0925 02:46:19.569577 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:46:19.570007 (+ 430us) server_negotiation.cc:407] Connection header received
0925 02:46:19.570157 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:46:19.570159 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:46:19.570209 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:46:19.570276 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:46:19.571206 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:46:19.571993 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:46:19.572949 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:46:19.573293 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:46:19.573398 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:46:19.574807 (+ 1409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:46:19.574811 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:46:19.574813 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:46:19.574840 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:46:19.576223 (+ 1383us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:46:19.576620 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:46:19.576623 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:46:19.576624 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:46:19.576716 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:46:19.576982 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:46:19.576986 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:46:19.576987 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:46:19.577202 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:46:19.577415 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0925 02:46:19.577528 (+ 113us) server_negotiation.cc:299] Negotiation successful
0925 02:46:19.577623 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
Sep 25 02:46:19 dist-test-slave-kznw krb5kdc[29489](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304379, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.131@KRBTEST.COM for kudu/127.13.157.190@KRBTEST.COM
I20260925 02:46:19.581827 30544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:19.569458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.131:45627 (local address 127.13.157.190:41669)
0925 02:46:19.569826 (+ 368us) server_negotiation.cc:206] Beginning negotiation
0925 02:46:19.569829 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 02:46:19.570666 (+ 837us) server_negotiation.cc:407] Connection header received
0925 02:46:19.572160 (+ 1494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:46:19.572164 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:46:19.572214 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:46:19.572301 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:46:19.573546 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:46:19.574295 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:46:19.575737 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:46:19.575965 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:46:19.576047 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:46:19.578648 (+ 2601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:46:19.578652 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:46:19.578654 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:46:19.578680 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:46:19.580004 (+ 1324us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:46:19.580447 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:46:19.580450 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:46:19.580452 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:46:19.580529 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:46:19.580835 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:46:19.580839 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:46:19.580840 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:46:19.581003 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:46:19.581084 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0925 02:46:19.581545 (+ 461us) server_negotiation.cc:299] Negotiation successful
0925 02:46:19.581662 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":129,"server-negotiator.queue_time_us":303,"thread_start_us":98,"threads_started":1}
I20260925 02:46:19.582571 30545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:19.570036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:41669 (local address 127.13.157.131:45627)
0925 02:46:19.570540 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0925 02:46:19.570580 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0925 02:46:19.571942 (+ 1362us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:46:19.572452 (+ 510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:46:19.572460 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:46:19.572813 (+ 353us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:46:19.573406 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:46:19.573425 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:46:19.574415 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:46:19.574417 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:46:19.575618 (+ 1201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:46:19.575630 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:46:19.576110 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:46:19.576115 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:46:19.576316 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:46:19.576799 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:46:19.576819 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:46:19.578492 (+ 1673us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:46:19.580107 (+ 1615us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:46:19.580112 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:46:19.580126 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:46:19.580358 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:46:19.580610 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:46:19.580613 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:46:19.580615 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:46:19.580751 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:46:19.581098 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:46:19.581104 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:46:19.581326 (+ 222us) client_negotiation.cc:769] Sending connection context
0925 02:46:19.581486 (+ 160us) client_negotiation.cc:240] Negotiation successful
0925 02:46:19.581798 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":117,"threads_started":1}
I20260925 02:46:19.583578 30541 heartbeater.cc:347] Connected to a master server at 127.13.157.190:41669
I20260925 02:46:19.583775 30541 heartbeater.cc:464] Registering TS with master...
I20260925 02:46:19.584218 30541 heartbeater.cc:511] Master 127.13.157.190:41669 requested a full tablet report, sending...
I20260925 02:46:19.585115 30079 ts_manager.cc:194] Registered new tserver with Master: 519a5d55415b4a5aa62e9ada1250a9ec (127.13.157.131:44627)
I20260925 02:46:19.585628 30079 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.157.131@KRBTEST.COM'} at 127.13.157.131:45627
I20260925 02:46:19.591082 13942 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 02:46:19.594769 30544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:19.591620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44902 (local address 127.13.157.190:41669)
0925 02:46:19.591745 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0925 02:46:19.591749 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:46:19.591934 (+ 185us) server_negotiation.cc:407] Connection header received
0925 02:46:19.592067 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:46:19.592070 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:46:19.592178 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:46:19.592256 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 02:46:19.592428 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:46:19.592931 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:46:19.593787 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:46:19.594041 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:46:19.594117 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:46:19.594348 (+ 231us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 02:46:19.594488 (+ 140us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0925 02:46:19.594675 (+ 187us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:44902: token expired
Metrics: {"server-negotiator.queue_time_us":49}
W20260925 02:46:19.594816 30544 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:44902: token expired
W20260925 02:46:19.595293 30004 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.13.157.190:41669)
I20260925 02:46:19.601408 30544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:19.595615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44906 (local address 127.13.157.190:41669)
0925 02:46:19.595735 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0925 02:46:19.595738 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 02:46:19.595748 (+ 10us) server_negotiation.cc:407] Connection header received
0925 02:46:19.595851 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:46:19.595853 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:46:19.595887 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:46:19.595945 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:46:19.596523 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:46:19.596984 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:46:19.597667 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:46:19.597851 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:46:19.597913 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:46:19.599006 (+ 1093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:46:19.599009 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:46:19.599011 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:46:19.599031 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:46:19.600275 (+ 1244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:46:19.600595 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:46:19.600597 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:46:19.600598 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:46:19.600682 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:46:19.600856 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:46:19.600857 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:46:19.600858 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:46:19.600997 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:46:19.601087 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0925 02:46:19.601225 (+ 138us) server_negotiation.cc:299] Negotiation successful
0925 02:46:19.601306 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260925 02:46:19.603572 30079 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44906:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 02:46:19.606638 30079 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 02:46:19.613801 30202 tablet_service.cc:1467] Processing CreateTablet for tablet d70e788499bd47c29a5795583b21ca32 (DEFAULT_TABLE table=test-table [id=fd5b94ae85ee47518eab8b6c27a63bbb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:46:19.614447 30202 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d70e788499bd47c29a5795583b21ca32. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:46:19.617429 30160 tablet_bootstrap.cc:492] T d70e788499bd47c29a5795583b21ca32 P 4b0601a8350a44b9a3e871bdf175281b: Bootstrap starting.
I20260925 02:46:19.617703 30550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:19.613007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.130:45427 (local address 127.0.0.1:54800)
0925 02:46:19.613252 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0925 02:46:19.613264 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 02:46:19.613347 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:46:19.614162 (+ 815us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:46:19.614165 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:46:19.614183 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:46:19.614282 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:46:19.614288 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:46:19.615721 (+ 1433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:46:19.615724 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:46:19.616483 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:46:19.616493 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:46:19.617253 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:46:19.617257 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:46:19.617375 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:46:19.617419 (+ 44us) client_negotiation.cc:769] Sending connection context
0925 02:46:19.617482 (+ 63us) client_negotiation.cc:240] Negotiation successful
0925 02:46:19.617551 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":92,"threads_started":1}
I20260925 02:46:19.618011 30275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:19.612763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.131:44627 (local address 127.0.0.1:46668)
0925 02:46:19.612898 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0925 02:46:19.612913 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 02:46:19.612988 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:46:19.613979 (+ 991us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:46:19.613983 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:46:19.614003 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:46:19.614097 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:46:19.614103 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:46:19.615426 (+ 1323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:46:19.615429 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:46:19.616564 (+ 1135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:46:19.616574 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:46:19.617626 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:46:19.617630 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:46:19.617753 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:46:19.617776 (+ 23us) client_negotiation.cc:769] Sending connection context
0925 02:46:19.617826 (+ 50us) client_negotiation.cc:240] Negotiation successful
0925 02:46:19.617899 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62}
I20260925 02:46:19.618372 30160 tablet_bootstrap.cc:654] T d70e788499bd47c29a5795583b21ca32 P 4b0601a8350a44b9a3e871bdf175281b: Neither blocks nor log segments found. Creating new log.
I20260925 02:46:19.618685 30551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:19.612972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46668 (local address 127.13.157.131:44627)
0925 02:46:19.613465 (+ 493us) server_negotiation.cc:206] Beginning negotiation
0925 02:46:19.613474 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 02:46:19.613493 (+ 19us) server_negotiation.cc:407] Connection header received
0925 02:46:19.613559 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:46:19.613565 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:46:19.613800 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:46:19.613984 (+ 184us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:46:19.614234 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:46:19.615301 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:46:19.616722 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:46:19.617507 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:46:19.618283 (+ 776us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:46:19.618411 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0925 02:46:19.618460 (+ 49us) server_negotiation.cc:299] Negotiation successful
0925 02:46:19.618546 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":403,"thread_start_us":177,"threads_started":1}
I20260925 02:46:19.618714 30552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:19.613393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54800 (local address 127.13.157.130:45427)
0925 02:46:19.613723 (+ 330us) server_negotiation.cc:206] Beginning negotiation
0925 02:46:19.613733 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 02:46:19.613754 (+ 21us) server_negotiation.cc:407] Connection header received
0925 02:46:19.613823 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:46:19.613828 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:46:19.614005 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:46:19.614403 (+ 398us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:46:19.614594 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:46:19.615621 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:46:19.616601 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:46:19.617146 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:46:19.617254 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:46:19.618403 (+ 1149us) server_negotiation.cc:1035] Waiting for connection context
0925 02:46:19.618458 (+ 55us) server_negotiation.cc:299] Negotiation successful
0925 02:46:19.618555 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":232,"thread_start_us":121,"threads_started":1}
I20260925 02:46:19.621618 30474 tablet_service.cc:1467] Processing CreateTablet for tablet d70e788499bd47c29a5795583b21ca32 (DEFAULT_TABLE table=test-table [id=fd5b94ae85ee47518eab8b6c27a63bbb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:46:19.621620 30340 tablet_service.cc:1467] Processing CreateTablet for tablet d70e788499bd47c29a5795583b21ca32 (DEFAULT_TABLE table=test-table [id=fd5b94ae85ee47518eab8b6c27a63bbb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:46:19.622723 30340 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d70e788499bd47c29a5795583b21ca32. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:46:19.622728 30474 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d70e788499bd47c29a5795583b21ca32. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:46:19.623327 30160 tablet_bootstrap.cc:492] T d70e788499bd47c29a5795583b21ca32 P 4b0601a8350a44b9a3e871bdf175281b: No bootstrap required, opened a new log
I20260925 02:46:19.623415 30160 ts_tablet_manager.cc:1397] T d70e788499bd47c29a5795583b21ca32 P 4b0601a8350a44b9a3e871bdf175281b: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.001s
I20260925 02:46:19.623822 30160 raft_consensus.cc:348] T d70e788499bd47c29a5795583b21ca32 P 4b0601a8350a44b9a3e871bdf175281b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "519a5d55415b4a5aa62e9ada1250a9ec" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 44627 } } peers { permanent_uuid: "0fb2fcd1a5744120ae6146cbc623af16" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 45427 } } peers { permanent_uuid: "4b0601a8350a44b9a3e871bdf175281b" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 44449 } }
I20260925 02:46:19.623924 30160 raft_consensus.cc:374] T d70e788499bd47c29a5795583b21ca32 P 4b0601a8350a44b9a3e871bdf175281b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:46:19.623960 30160 raft_consensus.cc:729] T d70e788499bd47c29a5795583b21ca32 P 4b0601a8350a44b9a3e871bdf175281b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b0601a8350a44b9a3e871bdf175281b, State: Initialized, Role: FOLLOWER
I20260925 02:46:19.624080 30160 consensus_queue.cc:260] T d70e788499bd47c29a5795583b21ca32 P 4b0601a8350a44b9a3e871bdf175281b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "519a5d55415b4a5aa62e9ada1250a9ec" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 44627 } } peers { permanent_uuid: "0fb2fcd1a5744120ae6146cbc623af16" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 45427 } } peers { permanent_uuid: "4b0601a8350a44b9a3e871bdf175281b" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 44449 } }
I20260925 02:46:19.624333 30160 ts_tablet_manager.cc:1428] T d70e788499bd47c29a5795583b21ca32 P 4b0601a8350a44b9a3e871bdf175281b: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260925 02:46:19.631031 30555 tablet_bootstrap.cc:492] T d70e788499bd47c29a5795583b21ca32 P 519a5d55415b4a5aa62e9ada1250a9ec: Bootstrap starting.
I20260925 02:46:19.631031 30554 tablet_bootstrap.cc:492] T d70e788499bd47c29a5795583b21ca32 P 0fb2fcd1a5744120ae6146cbc623af16: Bootstrap starting.
I20260925 02:46:19.633687 30555 tablet_bootstrap.cc:654] T d70e788499bd47c29a5795583b21ca32 P 519a5d55415b4a5aa62e9ada1250a9ec: Neither blocks nor log segments found. Creating new log.
I20260925 02:46:19.633690 30554 tablet_bootstrap.cc:654] T d70e788499bd47c29a5795583b21ca32 P 0fb2fcd1a5744120ae6146cbc623af16: Neither blocks nor log segments found. Creating new log.
I20260925 02:46:19.634579 30554 log.cc:824] T d70e788499bd47c29a5795583b21ca32 P 0fb2fcd1a5744120ae6146cbc623af16: Log is configured to *not* fsync() on all Append() calls
I20260925 02:46:19.634579 30555 log.cc:824] T d70e788499bd47c29a5795583b21ca32 P 519a5d55415b4a5aa62e9ada1250a9ec: Log is configured to *not* fsync() on all Append() calls
I20260925 02:46:19.636507 30554 tablet_bootstrap.cc:492] T d70e788499bd47c29a5795583b21ca32 P 0fb2fcd1a5744120ae6146cbc623af16: No bootstrap required, opened a new log
I20260925 02:46:19.636528 30555 tablet_bootstrap.cc:492] T d70e788499bd47c29a5795583b21ca32 P 519a5d55415b4a5aa62e9ada1250a9ec: No bootstrap required, opened a new log
I20260925 02:46:19.636677 30554 ts_tablet_manager.cc:1397] T d70e788499bd47c29a5795583b21ca32 P 0fb2fcd1a5744120ae6146cbc623af16: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.002s
I20260925 02:46:19.636711 30555 ts_tablet_manager.cc:1397] T d70e788499bd47c29a5795583b21ca32 P 519a5d55415b4a5aa62e9ada1250a9ec: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.002s
I20260925 02:46:19.639484 30554 raft_consensus.cc:348] T d70e788499bd47c29a5795583b21ca32 P 0fb2fcd1a5744120ae6146cbc623af16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "519a5d55415b4a5aa62e9ada1250a9ec" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 44627 } } peers { permanent_uuid: "0fb2fcd1a5744120ae6146cbc623af16" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 45427 } } peers { permanent_uuid: "4b0601a8350a44b9a3e871bdf175281b" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 44449 } }
I20260925 02:46:19.639484 30555 raft_consensus.cc:348] T d70e788499bd47c29a5795583b21ca32 P 519a5d55415b4a5aa62e9ada1250a9ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "519a5d55415b4a5aa62e9ada1250a9ec" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 44627 } } peers { permanent_uuid: "0fb2fcd1a5744120ae6146cbc623af16" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 45427 } } peers { permanent_uuid: "4b0601a8350a44b9a3e871bdf175281b" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 44449 } }
I20260925 02:46:19.639708 30554 raft_consensus.cc:374] T d70e788499bd47c29a5795583b21ca32 P 0fb2fcd1a5744120ae6146cbc623af16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:46:19.639717 30555 raft_consensus.cc:374] T d70e788499bd47c29a5795583b21ca32 P 519a5d55415b4a5aa62e9ada1250a9ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:46:19.639755 30554 raft_consensus.cc:729] T d70e788499bd47c29a5795583b21ca32 P 0fb2fcd1a5744120ae6146cbc623af16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fb2fcd1a5744120ae6146cbc623af16, State: Initialized, Role: FOLLOWER
I20260925 02:46:19.639776 30555 raft_consensus.cc:729] T d70e788499bd47c29a5795583b21ca32 P 519a5d55415b4a5aa62e9ada1250a9ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 519a5d55415b4a5aa62e9ada1250a9ec, State: Initialized, Role: FOLLOWER
I20260925 02:46:19.640166 30555 consensus_queue.cc:260] T d70e788499bd47c29a5795583b21ca32 P 519a5d55415b4a5aa62e9ada1250a9ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "519a5d55415b4a5aa62e9ada1250a9ec" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 44627 } } peers { permanent_uuid: "0fb2fcd1a5744120ae6146cbc623af16" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 45427 } } peers { permanent_uuid: "4b0601a8350a44b9a3e871bdf175281b" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 44449 } }
I20260925 02:46:19.640196 30554 consensus_queue.cc:260] T d70e788499bd47c29a5795583b21ca32 P 0fb2fcd1a5744120ae6146cbc623af16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "519a5d55415b4a5aa62e9ada1250a9ec" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 44627 } } peers { permanent_uuid: "0fb2fcd1a5744120ae6146cbc623af16" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 45427 } } peers { permanent_uuid: "4b0601a8350a44b9a3e871bdf175281b" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 44449 } }
I20260925 02:46:19.640775 30541 heartbeater.cc:503] Master 127.13.157.190:41669 was elected leader, sending a full tablet report...
I20260925 02:46:19.640779 30407 heartbeater.cc:503] Master 127.13.157.190:41669 was elected leader, sending a full tablet report...
I20260925 02:46:19.641134 30555 ts_tablet_manager.cc:1428] T d70e788499bd47c29a5795583b21ca32 P 519a5d55415b4a5aa62e9ada1250a9ec: Time spent starting tablet: real 0.004s user 0.004s sys 0.001s
I20260925 02:46:19.641134 30554 ts_tablet_manager.cc:1428] T d70e788499bd47c29a5795583b21ca32 P 0fb2fcd1a5744120ae6146cbc623af16: Time spent starting tablet: real 0.004s user 0.004s sys 0.001s
I20260925 02:46:19.647329 30559 raft_consensus.cc:482] T d70e788499bd47c29a5795583b21ca32 P 0fb2fcd1a5744120ae6146cbc623af16 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 02:46:19.647470 30559 raft_consensus.cc:504] T d70e788499bd47c29a5795583b21ca32 P 0fb2fcd1a5744120ae6146cbc623af16 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "519a5d55415b4a5aa62e9ada1250a9ec" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 44627 } } peers { permanent_uuid: "0fb2fcd1a5744120ae6146cbc623af16" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 45427 } } peers { permanent_uuid: "4b0601a8350a44b9a3e871bdf175281b" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 44449 } }
I20260925 02:46:19.648461 30559 leader_election.cc:288] T d70e788499bd47c29a5795583b21ca32 P 0fb2fcd1a5744120ae6146cbc623af16 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 519a5d55415b4a5aa62e9ada1250a9ec (127.13.157.131:44627), 4b0601a8350a44b9a3e871bdf175281b (127.13.157.129:44449)
I20260925 02:46:19.652130 30409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:19.648658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.131:44627 (local address 127.13.157.130:43897)
0925 02:46:19.648863 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0925 02:46:19.648878 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 02:46:19.648969 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:46:19.649174 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:46:19.649177 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:46:19.649192 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:46:19.649297 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:46:19.649302 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:46:19.649973 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:46:19.649976 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:46:19.651019 (+ 1043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:46:19.651030 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:46:19.651755 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:46:19.651758 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:46:19.651838 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:46:19.651859 (+ 21us) client_negotiation.cc:769] Sending connection context
0925 02:46:19.651940 (+ 81us) client_negotiation.cc:240] Negotiation successful
0925 02:46:19.652018 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133}
I20260925 02:46:19.652155 30551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:19.648731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.130:43897 (local address 127.13.157.131:44627)
0925 02:46:19.648857 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0925 02:46:19.648862 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:46:19.648932 (+ 70us) server_negotiation.cc:407] Connection header received
0925 02:46:19.649059 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:46:19.649062 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:46:19.649102 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:46:19.649168 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:46:19.649388 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:46:19.649878 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:46:19.651118 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:46:19.651653 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:46:19.651769 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:46:19.651819 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0925 02:46:19.651959 (+ 140us) server_negotiation.cc:299] Negotiation successful
0925 02:46:19.652025 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260925 02:46:19.652526 30560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:19.648682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.129:44449 (local address 127.13.157.130:60191)
0925 02:46:19.649033 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0925 02:46:19.649043 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 02:46:19.649134 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:46:19.649376 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:46:19.649378 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:46:19.649393 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:46:19.649506 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:46:19.649511 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:46:19.650233 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:46:19.650236 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:46:19.651286 (+ 1050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:46:19.651296 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:46:19.652136 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:46:19.652140 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:46:19.652240 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:46:19.652265 (+ 25us) client_negotiation.cc:769] Sending connection context
0925 02:46:19.652329 (+ 64us) client_negotiation.cc:240] Negotiation successful
0925 02:46:19.652396 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"mutex_wait_us":3,"thread_start_us":110,"threads_started":1}
I20260925 02:46:19.652529 30276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:19.648793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.130:60191 (local address 127.13.157.129:44449)
0925 02:46:19.648900 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0925 02:46:19.648905 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:46:19.649113 (+ 208us) server_negotiation.cc:407] Connection header received
0925 02:46:19.649227 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:46:19.649230 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:46:19.649283 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:46:19.649360 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:46:19.649600 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:46:19.650132 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:46:19.651400 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:46:19.652034 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:46:19.652145 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:46:19.652319 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0925 02:46:19.652354 (+ 35us) server_negotiation.cc:299] Negotiation successful
0925 02:46:19.652417 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260925 02:46:19.652673 30494 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d70e788499bd47c29a5795583b21ca32" candidate_uuid: "0fb2fcd1a5744120ae6146cbc623af16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "519a5d55415b4a5aa62e9ada1250a9ec" is_pre_election: true
I20260925 02:46:19.652930 30494 raft_consensus.cc:2454] T d70e788499bd47c29a5795583b21ca32 P 519a5d55415b4a5aa62e9ada1250a9ec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0fb2fcd1a5744120ae6146cbc623af16 in term 0.
I20260925 02:46:19.652989 30222 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d70e788499bd47c29a5795583b21ca32" candidate_uuid: "0fb2fcd1a5744120ae6146cbc623af16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b0601a8350a44b9a3e871bdf175281b" is_pre_election: true
I20260925 02:46:19.653268 30222 raft_consensus.cc:2454] T d70e788499bd47c29a5795583b21ca32 P 4b0601a8350a44b9a3e871bdf175281b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0fb2fcd1a5744120ae6146cbc623af16 in term 0.
I20260925 02:46:19.653386 30293 leader_election.cc:302] T d70e788499bd47c29a5795583b21ca32 P 0fb2fcd1a5744120ae6146cbc623af16 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0fb2fcd1a5744120ae6146cbc623af16, 519a5d55415b4a5aa62e9ada1250a9ec; no voters:
I20260925 02:46:19.653657 30559 raft_consensus.cc:2781] T d70e788499bd47c29a5795583b21ca32 P 0fb2fcd1a5744120ae6146cbc623af16 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 02:46:19.653772 30559 raft_consensus.cc:482] T d70e788499bd47c29a5795583b21ca32 P 0fb2fcd1a5744120ae6146cbc623af16 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 02:46:19.653831 30559 raft_consensus.cc:3037] T d70e788499bd47c29a5795583b21ca32 P 0fb2fcd1a5744120ae6146cbc623af16 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:46:19.654806 30559 raft_consensus.cc:504] T d70e788499bd47c29a5795583b21ca32 P 0fb2fcd1a5744120ae6146cbc623af16 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "519a5d55415b4a5aa62e9ada1250a9ec" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 44627 } } peers { permanent_uuid: "0fb2fcd1a5744120ae6146cbc623af16" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 45427 } } peers { permanent_uuid: "4b0601a8350a44b9a3e871bdf175281b" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 44449 } }
I20260925 02:46:19.655150 30559 leader_election.cc:288] T d70e788499bd47c29a5795583b21ca32 P 0fb2fcd1a5744120ae6146cbc623af16 [CANDIDATE]: Term 1 election: Requested vote from peers 519a5d55415b4a5aa62e9ada1250a9ec (127.13.157.131:44627), 4b0601a8350a44b9a3e871bdf175281b (127.13.157.129:44449)
I20260925 02:46:19.655462 30222 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d70e788499bd47c29a5795583b21ca32" candidate_uuid: "0fb2fcd1a5744120ae6146cbc623af16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b0601a8350a44b9a3e871bdf175281b"
I20260925 02:46:19.655576 30222 raft_consensus.cc:3037] T d70e788499bd47c29a5795583b21ca32 P 4b0601a8350a44b9a3e871bdf175281b [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:46:19.656581 30222 raft_consensus.cc:2454] T d70e788499bd47c29a5795583b21ca32 P 4b0601a8350a44b9a3e871bdf175281b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0fb2fcd1a5744120ae6146cbc623af16 in term 1.
I20260925 02:46:19.656821 30494 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d70e788499bd47c29a5795583b21ca32" candidate_uuid: "0fb2fcd1a5744120ae6146cbc623af16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "519a5d55415b4a5aa62e9ada1250a9ec"
I20260925 02:46:19.656857 30295 leader_election.cc:302] T d70e788499bd47c29a5795583b21ca32 P 0fb2fcd1a5744120ae6146cbc623af16 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0fb2fcd1a5744120ae6146cbc623af16, 4b0601a8350a44b9a3e871bdf175281b; no voters:
I20260925 02:46:19.656927 30494 raft_consensus.cc:3037] T d70e788499bd47c29a5795583b21ca32 P 519a5d55415b4a5aa62e9ada1250a9ec [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:46:19.656998 30559 raft_consensus.cc:2781] T d70e788499bd47c29a5795583b21ca32 P 0fb2fcd1a5744120ae6146cbc623af16 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:46:19.657226 30559 raft_consensus.cc:686] T d70e788499bd47c29a5795583b21ca32 P 0fb2fcd1a5744120ae6146cbc623af16 [term 1 LEADER]: Becoming Leader. State: Replica: 0fb2fcd1a5744120ae6146cbc623af16, State: Running, Role: LEADER
I20260925 02:46:19.657567 30559 consensus_queue.cc:237] T d70e788499bd47c29a5795583b21ca32 P 0fb2fcd1a5744120ae6146cbc623af16 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "519a5d55415b4a5aa62e9ada1250a9ec" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 44627 } } peers { permanent_uuid: "0fb2fcd1a5744120ae6146cbc623af16" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 45427 } } peers { permanent_uuid: "4b0601a8350a44b9a3e871bdf175281b" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 44449 } }
I20260925 02:46:19.658155 30494 raft_consensus.cc:2454] T d70e788499bd47c29a5795583b21ca32 P 519a5d55415b4a5aa62e9ada1250a9ec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0fb2fcd1a5744120ae6146cbc623af16 in term 1.
I20260925 02:46:19.661110 30079 catalog_manager.cc:5495] T d70e788499bd47c29a5795583b21ca32 P 0fb2fcd1a5744120ae6146cbc623af16 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0fb2fcd1a5744120ae6146cbc623af16 (127.13.157.130). New cstate: current_term: 1 leader_uuid: "0fb2fcd1a5744120ae6146cbc623af16" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "519a5d55415b4a5aa62e9ada1250a9ec" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 44627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0fb2fcd1a5744120ae6146cbc623af16" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 45427 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b0601a8350a44b9a3e871bdf175281b" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 44449 } health_report { overall_health: UNKNOWN } } }
I20260925 02:46:19.681483 30552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:19.678323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54806 (local address 127.13.157.130:45427)
0925 02:46:19.678443 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0925 02:46:19.678446 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 02:46:19.678459 (+ 13us) server_negotiation.cc:407] Connection header received
0925 02:46:19.678571 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:46:19.678573 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:46:19.678629 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:46:19.678724 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 02:46:19.678916 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:46:19.679450 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:46:19.680246 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:46:19.680454 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:46:19.680531 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:46:19.680766 (+ 235us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 02:46:19.681184 (+ 418us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 02:46:19.681272 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0925 02:46:19.681299 (+ 27us) server_negotiation.cc:299] Negotiation successful
0925 02:46:19.681349 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260925 02:46:19.687300 30222 raft_consensus.cc:1261] T d70e788499bd47c29a5795583b21ca32 P 4b0601a8350a44b9a3e871bdf175281b [term 1 FOLLOWER]: Refusing update from remote peer 0fb2fcd1a5744120ae6146cbc623af16: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:46:19.687304 30494 raft_consensus.cc:1261] T d70e788499bd47c29a5795583b21ca32 P 519a5d55415b4a5aa62e9ada1250a9ec [term 1 FOLLOWER]: Refusing update from remote peer 0fb2fcd1a5744120ae6146cbc623af16: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:46:19.687994 30561 consensus_queue.cc:1035] T d70e788499bd47c29a5795583b21ca32 P 0fb2fcd1a5744120ae6146cbc623af16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b0601a8350a44b9a3e871bdf175281b" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 44449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:46:19.688176 30559 consensus_queue.cc:1035] T d70e788499bd47c29a5795583b21ca32 P 0fb2fcd1a5744120ae6146cbc623af16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "519a5d55415b4a5aa62e9ada1250a9ec" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 44627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260925 02:46:19.693576 30269 tablet.cc:2369] T d70e788499bd47c29a5795583b21ca32 P 4b0601a8350a44b9a3e871bdf175281b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 02:46:19.697227 30566 mvcc.cc:205] Tried to move back new op lower bound from 7333086739192168448 to 7333086739080581120. Current Snapshot: MvccSnapshot[applied={T|T < 7333086739192168448}]
I20260925 02:46:19.702503 30079 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44906:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 02:46:19.702657 30079 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44906:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 02:46:19.705008 30079 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 350851ecf2bf4219bc7578b7de80ffc1: Sending DeleteTablet for 3 replicas of tablet d70e788499bd47c29a5795583b21ca32
I20260925 02:46:19.705760 30340 tablet_service.cc:1514] Processing DeleteTablet for tablet d70e788499bd47c29a5795583b21ca32 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:46:19 UTC) from {username='kudu', principal='kudu/127.13.157.190@KRBTEST.COM'} at 127.0.0.1:54800
I20260925 02:46:19.705746 30474 tablet_service.cc:1514] Processing DeleteTablet for tablet d70e788499bd47c29a5795583b21ca32 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:46:19 UTC) from {username='kudu', principal='kudu/127.13.157.190@KRBTEST.COM'} at 127.0.0.1:46668
I20260925 02:46:19.705945 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 30118
I20260925 02:46:19.705988 30202 tablet_service.cc:1514] Processing DeleteTablet for tablet d70e788499bd47c29a5795583b21ca32 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:46:19 UTC) from {username='kudu', principal='kudu/127.13.157.190@KRBTEST.COM'} at 127.0.0.1:56970
I20260925 02:46:19.707350 30574 tablet_replica.cc:330] T d70e788499bd47c29a5795583b21ca32 P 0fb2fcd1a5744120ae6146cbc623af16: stopping tablet replica
I20260925 02:46:19.707650 30574 raft_consensus.cc:2229] T d70e788499bd47c29a5795583b21ca32 P 0fb2fcd1a5744120ae6146cbc623af16 [term 1 LEADER]: Raft consensus shutting down.
I20260925 02:46:19.708016 30574 raft_consensus.cc:2258] T d70e788499bd47c29a5795583b21ca32 P 0fb2fcd1a5744120ae6146cbc623af16 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:46:19.708379 30575 tablet_replica.cc:330] T d70e788499bd47c29a5795583b21ca32 P 519a5d55415b4a5aa62e9ada1250a9ec: stopping tablet replica
I20260925 02:46:19.708671 30575 raft_consensus.cc:2229] T d70e788499bd47c29a5795583b21ca32 P 519a5d55415b4a5aa62e9ada1250a9ec [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 02:46:19.708987 30575 raft_consensus.cc:2258] T d70e788499bd47c29a5795583b21ca32 P 519a5d55415b4a5aa62e9ada1250a9ec [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:46:19.709852 30574 ts_tablet_manager.cc:1905] T d70e788499bd47c29a5795583b21ca32 P 0fb2fcd1a5744120ae6146cbc623af16: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:46:19.710471 30575 ts_tablet_manager.cc:1905] T d70e788499bd47c29a5795583b21ca32 P 519a5d55415b4a5aa62e9ada1250a9ec: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:46:19.712839 30574 ts_tablet_manager.cc:1918] T d70e788499bd47c29a5795583b21ca32 P 0fb2fcd1a5744120ae6146cbc623af16: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 02:46:19.713020 30574 log.cc:1196] T d70e788499bd47c29a5795583b21ca32 P 0fb2fcd1a5744120ae6146cbc623af16: Deleting WAL directory at /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-1/wal/wals/d70e788499bd47c29a5795583b21ca32
W20260925 02:46:19.713064 30064 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.157.129:44449 (error 108)
I20260925 02:46:19.713482 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 30272
I20260925 02:46:19.713505 30574 ts_tablet_manager.cc:1939] T d70e788499bd47c29a5795583b21ca32 P 0fb2fcd1a5744120ae6146cbc623af16: Deleting consensus metadata
I20260925 02:46:19.714584 30575 ts_tablet_manager.cc:1918] T d70e788499bd47c29a5795583b21ca32 P 519a5d55415b4a5aa62e9ada1250a9ec: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 02:46:19.714679 30575 log.cc:1196] T d70e788499bd47c29a5795583b21ca32 P 519a5d55415b4a5aa62e9ada1250a9ec: Deleting WAL directory at /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-2/wal/wals/d70e788499bd47c29a5795583b21ca32
I20260925 02:46:19.714974 30575 ts_tablet_manager.cc:1939] T d70e788499bd47c29a5795583b21ca32 P 519a5d55415b4a5aa62e9ada1250a9ec: Deleting consensus metadata
I20260925 02:46:19.716018 30275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:19.715692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.129:44449 (local address 127.0.0.1:56986)
0925 02:46:19.715845 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0925 02:46:19.715925 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.129:44449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260925 02:46:19.716387 30062 catalog_manager.cc:4841] TS 519a5d55415b4a5aa62e9ada1250a9ec (127.13.157.131:44627): tablet d70e788499bd47c29a5795583b21ca32 (table test-table [id=fd5b94ae85ee47518eab8b6c27a63bbb]) successfully deleted
W20260925 02:46:19.716248 30064 catalog_manager.cc:4584] TS 4b0601a8350a44b9a3e871bdf175281b (127.13.157.129:44449): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d70e788499bd47c29a5795583b21ca32: Network error: Client connection negotiation failed: client connection to 127.13.157.129:44449: connect: Connection refused (error 111)
I20260925 02:46:19.719699 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 30410
I20260925 02:46:19.721552 30275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:19.721297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.130:45427 (local address 127.0.0.1:54818)
0925 02:46:19.721431 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0925 02:46:19.721478 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.130:45427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260925 02:46:19.725062 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 30045
I20260925 02:46:39.731519 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.157.190:41669
--webserver_interface=127.13.157.190
--webserver_port=34797
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab
--principal=kudu/127.13.157.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:38239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.157.190:41669
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:46:39.841393 30577 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:46:39.841661 30577 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:46:39.841717 30577 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:46:39.845743 30577 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260925 02:46:39.845808 30577 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260925 02:46:39.845830 30577 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260925 02:46:39.845846 30577 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:46:39.845870 30577 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:46:39.845899 30577 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:46:39.845929 30577 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:46:39.845956 30577 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:46:39.851323 30577 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:38239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.13.157.190:41669
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab
--principal=kudu/127.13.157.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.157.190:41669
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.157.190
--webserver_port=34797
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30577
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:46:39.852398 30577 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:46:39.853430 30577 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 02:46:39.862084 30577 server_base.cc:1034] running on GCE node
W20260925 02:46:39.862125 30583 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:46:39.862159 30586 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:46:39.862073 30584 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:46:39.862893 30577 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:46:39.864033 30577 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:46:39.865214 30577 hybrid_clock.cc:648] HybridClock initialized: now 1790304399865194 us; error 32 us; skew 500 ppm
Sep 25 02:46:39 dist-test-slave-kznw krb5kdc[29489](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304399, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:46:39.868175 30577 init.cc:377] Logged in from keytab as kudu/127.13.157.190@KRBTEST.COM (short username kudu)
I20260925 02:46:39.869133 30577 webserver.cc:460] Webserver started at http://127.13.157.190:34797/ using document root <none> and password file <none>
I20260925 02:46:39.869678 30577 fs_manager.cc:359] Metadata directory not provided
I20260925 02:46:39.869745 30577 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:46:39.873703 30577 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260925 02:46:39.875816 30593 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:46:39.877162 30577 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260925 02:46:39.877288 30577 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/master-0/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/master-0/wal
uuid: "350851ecf2bf4219bc7578b7de80ffc1"
format_stamp: "Formatted at 2026-09-25 02:45:57 on dist-test-slave-kznw"
server_key: "c3fe0d9e324b624e722df0418a656b85"
server_key_iv: "f23e3496dfb85ca0bcd7e73cb0a7c965"
server_key_version: "encryptionkey@0"
I20260925 02:46:39.877846 30577 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:46:39.907979 30577 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:46:39.911314 30577 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:46:39.911512 30577 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:46:39.918450 30577 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.190:41669
I20260925 02:46:39.918740 30645 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.190:41669 every 8 connection(s)
I20260925 02:46:39.919565 30577 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
I20260925 02:46:39.923969 30646 sys_catalog.cc:263] Verifying existing consensus state
I20260925 02:46:39.925602 30646 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 350851ecf2bf4219bc7578b7de80ffc1: Bootstrap starting.
I20260925 02:46:39.928490 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 30577
I20260925 02:46:39.928987 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.157.129:44449
--local_ip_for_outbound_sockets=127.13.157.129
--tserver_master_addrs=127.13.157.190:41669
--webserver_port=39511
--webserver_interface=127.13.157.129
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.129.keytab
--principal=kudu/127.13.157.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:38239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 02:46:39.943755 30646 log.cc:824] T 00000000000000000000000000000000 P 350851ecf2bf4219bc7578b7de80ffc1: Log is configured to *not* fsync() on all Append() calls
I20260925 02:46:39.955216 30646 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 350851ecf2bf4219bc7578b7de80ffc1: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 02:46:39.955590 30646 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 350851ecf2bf4219bc7578b7de80ffc1: Bootstrap complete.
I20260925 02:46:39.959442 30646 raft_consensus.cc:348] T 00000000000000000000000000000000 P 350851ecf2bf4219bc7578b7de80ffc1 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "350851ecf2bf4219bc7578b7de80ffc1" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 41669 } }
I20260925 02:46:39.959842 30646 raft_consensus.cc:729] T 00000000000000000000000000000000 P 350851ecf2bf4219bc7578b7de80ffc1 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 350851ecf2bf4219bc7578b7de80ffc1, State: Initialized, Role: FOLLOWER
I20260925 02:46:39.960336 30646 consensus_queue.cc:260] T 00000000000000000000000000000000 P 350851ecf2bf4219bc7578b7de80ffc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "350851ecf2bf4219bc7578b7de80ffc1" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 41669 } }
I20260925 02:46:39.960464 30646 raft_consensus.cc:388] T 00000000000000000000000000000000 P 350851ecf2bf4219bc7578b7de80ffc1 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 02:46:39.960527 30646 raft_consensus.cc:482] T 00000000000000000000000000000000 P 350851ecf2bf4219bc7578b7de80ffc1 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 02:46:39.960618 30646 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 350851ecf2bf4219bc7578b7de80ffc1 [term 2 FOLLOWER]: Advancing to term 3
I20260925 02:46:39.962479 30646 raft_consensus.cc:504] T 00000000000000000000000000000000 P 350851ecf2bf4219bc7578b7de80ffc1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "350851ecf2bf4219bc7578b7de80ffc1" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 41669 } }
I20260925 02:46:39.962807 30646 leader_election.cc:302] T 00000000000000000000000000000000 P 350851ecf2bf4219bc7578b7de80ffc1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 350851ecf2bf4219bc7578b7de80ffc1; no voters:
I20260925 02:46:39.963126 30646 leader_election.cc:288] T 00000000000000000000000000000000 P 350851ecf2bf4219bc7578b7de80ffc1 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260925 02:46:39.963368 30650 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 350851ecf2bf4219bc7578b7de80ffc1 [term 3 FOLLOWER]: Leader election won for term 3
I20260925 02:46:39.963591 30650 raft_consensus.cc:686] T 00000000000000000000000000000000 P 350851ecf2bf4219bc7578b7de80ffc1 [term 3 LEADER]: Becoming Leader. State: Replica: 350851ecf2bf4219bc7578b7de80ffc1, State: Running, Role: LEADER
I20260925 02:46:39.963896 30650 consensus_queue.cc:237] T 00000000000000000000000000000000 P 350851ecf2bf4219bc7578b7de80ffc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "350851ecf2bf4219bc7578b7de80ffc1" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 41669 } }
I20260925 02:46:39.964489 30646 sys_catalog.cc:564] T 00000000000000000000000000000000 P 350851ecf2bf4219bc7578b7de80ffc1 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:46:39.965363 30651 sys_catalog.cc:455] T 00000000000000000000000000000000 P 350851ecf2bf4219bc7578b7de80ffc1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "350851ecf2bf4219bc7578b7de80ffc1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "350851ecf2bf4219bc7578b7de80ffc1" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 41669 } } }
I20260925 02:46:39.965503 30651 sys_catalog.cc:458] T 00000000000000000000000000000000 P 350851ecf2bf4219bc7578b7de80ffc1 [sys.catalog]: This master's current role is: LEADER
I20260925 02:46:39.965885 30659 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:46:39.965878 30652 sys_catalog.cc:455] T 00000000000000000000000000000000 P 350851ecf2bf4219bc7578b7de80ffc1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 350851ecf2bf4219bc7578b7de80ffc1. Latest consensus state: current_term: 3 leader_uuid: "350851ecf2bf4219bc7578b7de80ffc1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "350851ecf2bf4219bc7578b7de80ffc1" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 41669 } } }
I20260925 02:46:39.965975 30652 sys_catalog.cc:458] T 00000000000000000000000000000000 P 350851ecf2bf4219bc7578b7de80ffc1 [sys.catalog]: This master's current role is: LEADER
I20260925 02:46:39.971122 30659 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:46:39.971802 30659 catalog_manager.cc:1259] Loaded cluster ID: 0f67716fe3564d8480b1345f47e8b83e
I20260925 02:46:39.971858 30659 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 02:46:39.973728 30659 catalog_manager.cc:1504] Loading token signing keys...
I20260925 02:46:39.974325 30659 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 350851ecf2bf4219bc7578b7de80ffc1: Loaded TSK: 0
I20260925 02:46:39.974392 30659 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 350851ecf2bf4219bc7578b7de80ffc1: Loaded TSK: 1
F20260925 02:46:39.974996 30659 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f231cb2f9ad google::LogMessage::Fail() at ??:0
@ 0x7f231cb3288a google::LogMessage::SendToLog() at ??:0
@ 0x7f231cb2f4de google::LogMessage::Flush() at ??:0
@ 0x7f231cb30a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f231f7f318d kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f231f7ebf0d kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f23274947bb kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f232746e99c kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f232746f178 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlvE3_clEv at ??:0
@ 0x7f23274b0df8 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f23274cf82b std::function<>::operator()() at ??:0
@ 0x7f232746eb4f _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f232746ff3e kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f232746c579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f23274b0640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f231ed813d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f231ed81c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f231ed833f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f231ed72512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f23203b66db start_thread at ??:0
@ 0x7f231b62d61f clone at ??:0
W20260925 02:46:40.076831 30648 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:46:40.077097 30648 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:46:40.077140 30648 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:46:40.081287 30648 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:46:40.081357 30648 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:46:40.081420 30648 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.157.129
I20260925 02:46:40.087045 30648 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:38239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.129.keytab
--principal=kudu/127.13.157.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.157.129:44449
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.157.129
--webserver_port=39511
--tserver_master_addrs=127.13.157.190:41669
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30648
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.157.129
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:46:40.088212 30648 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:46:40.089254 30648 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:46:40.099195 30693 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:46:40.099364 30694 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:46:40.099255 30648 server_base.cc:1034] running on GCE node
W20260925 02:46:40.099516 30696 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:46:40.099933 30648 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:46:40.100597 30648 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:46:40.101775 30648 hybrid_clock.cc:648] HybridClock initialized: now 1790304400101739 us; error 58 us; skew 500 ppm
Sep 25 02:46:40 dist-test-slave-kznw krb5kdc[29489](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304400, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:46:40.104769 30648 init.cc:377] Logged in from keytab as kudu/127.13.157.129@KRBTEST.COM (short username kudu)
I20260925 02:46:40.105690 30648 webserver.cc:460] Webserver started at http://127.13.157.129:39511/ using document root <none> and password file <none>
I20260925 02:46:40.106272 30648 fs_manager.cc:359] Metadata directory not provided
I20260925 02:46:40.106343 30648 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:46:40.110335 30648 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260925 02:46:40.112686 30703 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:46:40.114115 30648 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260925 02:46:40.114236 30648 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-0/wal
uuid: "4b0601a8350a44b9a3e871bdf175281b"
format_stamp: "Formatted at 2026-09-25 02:45:57 on dist-test-slave-kznw"
server_key: "9113657260e58d11ff87393cbb46c8ee"
server_key_iv: "fc69a696594b6b9a8cac4f13e8ba372f"
server_key_version: "encryptionkey@0"
I20260925 02:46:40.114820 30648 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:46:40.127128 30648 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:46:40.130178 30648 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:46:40.130347 30648 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:46:40.130941 30648 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:46:40.132047 30710 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260925 02:46:40.136086 30648 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260925 02:46:40.136143 30648 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.004s user 0.000s sys 0.000s
I20260925 02:46:40.136195 30648 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260925 02:46:40.137845 30648 ts_tablet_manager.cc:610] Registered 1 tablets
I20260925 02:46:40.137900 30648 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260925 02:46:40.138186 30710 tablet_bootstrap.cc:492] T d70e788499bd47c29a5795583b21ca32 P 4b0601a8350a44b9a3e871bdf175281b: Bootstrap starting.
I20260925 02:46:40.147980 30648 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.129:44449
I20260925 02:46:40.148275 30818 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.129:44449 every 8 connection(s)
I20260925 02:46:40.149225 30648 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-0/data/info.pb
I20260925 02:46:40.156497 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 30648
I20260925 02:46:40.157032 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.157.130:45427
--local_ip_for_outbound_sockets=127.13.157.130
--tserver_master_addrs=127.13.157.190:41669
--webserver_port=44459
--webserver_interface=127.13.157.130
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.130.keytab
--principal=kudu/127.13.157.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:38239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 02:46:40.157981 30710 log.cc:824] T d70e788499bd47c29a5795583b21ca32 P 4b0601a8350a44b9a3e871bdf175281b: Log is configured to *not* fsync() on all Append() calls
I20260925 02:46:40.167253 30710 tablet_bootstrap.cc:492] T d70e788499bd47c29a5795583b21ca32 P 4b0601a8350a44b9a3e871bdf175281b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 02:46:40.167681 30710 tablet_bootstrap.cc:492] T d70e788499bd47c29a5795583b21ca32 P 4b0601a8350a44b9a3e871bdf175281b: Bootstrap complete.
I20260925 02:46:40.169018 30710 ts_tablet_manager.cc:1397] T d70e788499bd47c29a5795583b21ca32 P 4b0601a8350a44b9a3e871bdf175281b: Time spent bootstrapping tablet: real 0.031s user 0.014s sys 0.015s
I20260925 02:46:40.172333 30710 raft_consensus.cc:348] T d70e788499bd47c29a5795583b21ca32 P 4b0601a8350a44b9a3e871bdf175281b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "519a5d55415b4a5aa62e9ada1250a9ec" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 44627 } } peers { permanent_uuid: "0fb2fcd1a5744120ae6146cbc623af16" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 45427 } } peers { permanent_uuid: "4b0601a8350a44b9a3e871bdf175281b" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 44449 } }
I20260925 02:46:40.172889 30710 raft_consensus.cc:729] T d70e788499bd47c29a5795583b21ca32 P 4b0601a8350a44b9a3e871bdf175281b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b0601a8350a44b9a3e871bdf175281b, State: Initialized, Role: FOLLOWER
I20260925 02:46:40.173446 30710 consensus_queue.cc:260] T d70e788499bd47c29a5795583b21ca32 P 4b0601a8350a44b9a3e871bdf175281b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "519a5d55415b4a5aa62e9ada1250a9ec" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 44627 } } peers { permanent_uuid: "0fb2fcd1a5744120ae6146cbc623af16" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 45427 } } peers { permanent_uuid: "4b0601a8350a44b9a3e871bdf175281b" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 44449 } }
I20260925 02:46:40.174606 30710 ts_tablet_manager.cc:1428] T d70e788499bd47c29a5795583b21ca32 P 4b0601a8350a44b9a3e871bdf175281b: Time spent starting tablet: real 0.006s user 0.006s sys 0.001s
W20260925 02:46:40.197180 30821 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:46:40.151563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:41669 (local address 127.13.157.129:43515)
0925 02:46:40.152149 (+ 586us) negotiation.cc:107] Waiting for socket to connect
0925 02:46:40.152209 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0925 02:46:40.153253 (+ 1044us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:46:40.196391 (+ 43138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:41669: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":149,"threads_started":1}
W20260925 02:46:40.197660 30819 heartbeater.cc:644] Failed to heartbeat to 127.13.157.190:41669 (0 consecutive failures): Network error: Failed to ping master at 127.13.157.190:41669: Client connection negotiation failed: client connection to 127.13.157.190:41669: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260925 02:46:40.198438 30821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:40.198166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:41669 (local address 127.13.157.129:40049)
0925 02:46:40.198312 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0925 02:46:40.198379 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:41669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260925 02:46:40.199079 30821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:40.198868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:41669 (local address 127.13.157.129:33099)
0925 02:46:40.198986 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0925 02:46:40.199031 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:41669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260925 02:46:40.199247 30819 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260925 02:46:40.269170 30823 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:46:40.269430 30823 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:46:40.269490 30823 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:46:40.273461 30823 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:46:40.273521 30823 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:46:40.273610 30823 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.157.130
I20260925 02:46:40.278739 30823 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:38239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.130.keytab
--principal=kudu/127.13.157.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.157.130:45427
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.157.130
--webserver_port=44459
--tserver_master_addrs=127.13.157.190:41669
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30823
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.157.130
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:46:40.279798 30823 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:46:40.280774 30823 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:46:40.290323 30832 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:46:40.290398 30823 server_base.cc:1034] running on GCE node
W20260925 02:46:40.290329 30829 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:46:40.290340 30830 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:46:40.291123 30823 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:46:40.291790 30823 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:46:40.292950 30823 hybrid_clock.cc:648] HybridClock initialized: now 1790304400292920 us; error 42 us; skew 500 ppm
Sep 25 02:46:40 dist-test-slave-kznw krb5kdc[29489](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304400, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:46:40.295761 30823 init.cc:377] Logged in from keytab as kudu/127.13.157.130@KRBTEST.COM (short username kudu)
I20260925 02:46:40.296706 30823 webserver.cc:460] Webserver started at http://127.13.157.130:44459/ using document root <none> and password file <none>
I20260925 02:46:40.297243 30823 fs_manager.cc:359] Metadata directory not provided
I20260925 02:46:40.297304 30823 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:46:40.301275 30823 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260925 02:46:40.303408 30839 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:46:40.304742 30823 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260925 02:46:40.304859 30823 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-1/wal
uuid: "0fb2fcd1a5744120ae6146cbc623af16"
format_stamp: "Formatted at 2026-09-25 02:45:58 on dist-test-slave-kznw"
server_key: "713b0d05899d454c583c3407fdc60fcc"
server_key_iv: "c85816a5865605371235f163aca80e25"
server_key_version: "encryptionkey@0"
I20260925 02:46:40.305404 30823 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:46:40.317291 30823 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:46:40.320513 30823 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:46:40.320690 30823 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:46:40.321273 30823 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:46:40.322343 30846 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
W20260925 02:46:40.326115 30846 ts_tablet_manager.cc:1852] T d70e788499bd47c29a5795583b21ca32 P 0fb2fcd1a5744120ae6146cbc623af16: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260925 02:46:40.326496 30846 ts_tablet_manager.cc:1905] T d70e788499bd47c29a5795583b21ca32 P 0fb2fcd1a5744120ae6146cbc623af16: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:46:40.330489 30846 ts_tablet_manager.cc:1918] T d70e788499bd47c29a5795583b21ca32 P 0fb2fcd1a5744120ae6146cbc623af16: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 02:46:40.330662 30846 ts_tablet_manager.cc:1939] T d70e788499bd47c29a5795583b21ca32 P 0fb2fcd1a5744120ae6146cbc623af16: Deleting consensus metadata
I20260925 02:46:40.330951 30823 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260925 02:46:40.331017 30823 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.009s user 0.000s sys 0.000s
I20260925 02:46:40.331095 30823 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:46:40.331121 30823 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:46:40.339648 30823 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.130:45427
I20260925 02:46:40.339743 30956 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.130:45427 every 8 connection(s)
I20260925 02:46:40.340543 30823 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-1/data/info.pb
I20260925 02:46:40.343776 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 30823
I20260925 02:46:40.344285 30959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:40.342733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:41669 (local address 127.13.157.130:46873)
0925 02:46:40.343251 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0925 02:46:40.343370 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:41669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":136,"threads_started":1}
I20260925 02:46:40.344333 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.157.131:44627
--local_ip_for_outbound_sockets=127.13.157.131
--tserver_master_addrs=127.13.157.190:41669
--webserver_port=36675
--webserver_interface=127.13.157.131
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.131.keytab
--principal=kudu/127.13.157.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:38239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:46:40.344805 30957 heartbeater.cc:644] Failed to heartbeat to 127.13.157.190:41669 (0 consecutive failures): Network error: Failed to ping master at 127.13.157.190:41669: Client connection negotiation failed: client connection to 127.13.157.190:41669: connect: Connection refused (error 111)
I20260925 02:46:40.345415 30959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:40.345148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:41669 (local address 127.13.157.130:46911)
0925 02:46:40.345281 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0925 02:46:40.345344 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:41669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260925 02:46:40.346146 30959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:40.345930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:41669 (local address 127.13.157.130:51039)
0925 02:46:40.346047 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0925 02:46:40.346095 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:41669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260925 02:46:40.346303 30957 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260925 02:46:40.453125 30960 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:46:40.453369 30960 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:46:40.453410 30960 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:46:40.457396 30960 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:46:40.457458 30960 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:46:40.457515 30960 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.157.131
I20260925 02:46:40.462725 30960 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:38239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.131.keytab
--principal=kudu/127.13.157.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.157.131:44627
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.157.131
--webserver_port=36675
--tserver_master_addrs=127.13.157.190:41669
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30960
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.157.131
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:46:40.463797 30960 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:46:40.464816 30960 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:46:40.474116 30965 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:46:40.474164 30968 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:46:40.474224 30966 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:46:40.474429 30960 server_base.cc:1034] running on GCE node
I20260925 02:46:40.474768 30960 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:46:40.475389 30960 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:46:40.476557 30960 hybrid_clock.cc:648] HybridClock initialized: now 1790304400476538 us; error 29 us; skew 500 ppm
Sep 25 02:46:40 dist-test-slave-kznw krb5kdc[29489](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304400, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:46:40.479231 30960 init.cc:377] Logged in from keytab as kudu/127.13.157.131@KRBTEST.COM (short username kudu)
I20260925 02:46:40.480083 30960 webserver.cc:460] Webserver started at http://127.13.157.131:36675/ using document root <none> and password file <none>
I20260925 02:46:40.480602 30960 fs_manager.cc:359] Metadata directory not provided
I20260925 02:46:40.480664 30960 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:46:40.484404 30960 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260925 02:46:40.486404 30975 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:46:40.487702 30960 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260925 02:46:40.487792 30960 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-2/wal
uuid: "519a5d55415b4a5aa62e9ada1250a9ec"
format_stamp: "Formatted at 2026-09-25 02:45:58 on dist-test-slave-kznw"
server_key: "0f842bb50e596776dba7eab124e59a30"
server_key_iv: "adeda174ae5b415696e7759553aad175"
server_key_version: "encryptionkey@0"
I20260925 02:46:40.488358 30960 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:46:40.499444 30960 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:46:40.502287 30960 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:46:40.502426 30960 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:46:40.502952 30960 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:46:40.503733 30960 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:46:40.503779 30960 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:46:40.503818 30960 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:46:40.503849 30960 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:46:40.512324 30960 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.131:44627
I20260925 02:46:40.512398 31089 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.131:44627 every 8 connection(s)
I20260925 02:46:40.513302 30960 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790304178325258-13942-0/minicluster-data/ts-2/data/info.pb
I20260925 02:46:40.517086 31092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:40.515409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:41669 (local address 127.13.157.131:48627)
0925 02:46:40.515851 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0925 02:46:40.515957 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:41669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":105,"threads_started":1}
W20260925 02:46:40.517601 31090 heartbeater.cc:644] Failed to heartbeat to 127.13.157.190:41669 (0 consecutive failures): Network error: Failed to ping master at 127.13.157.190:41669: Client connection negotiation failed: client connection to 127.13.157.190:41669: connect: Connection refused (error 111)
I20260925 02:46:40.518235 31092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:40.517957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:41669 (local address 127.13.157.131:47181)
0925 02:46:40.518054 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0925 02:46:40.518178 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:41669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260925 02:46:40.518803 31092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:40.518640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:41669 (local address 127.13.157.131:39533)
0925 02:46:40.518717 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0925 02:46:40.518757 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:41669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260925 02:46:40.519006 31090 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260925 02:46:40.520799 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 30960
I20260925 02:46:41.200294 31094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:41.199822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:41669 (local address 127.13.157.129:37219)
0925 02:46:41.200094 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0925 02:46:41.200177 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:41669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":74,"threads_started":1}
I20260925 02:46:41.347393 31095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:41.346954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:41669 (local address 127.13.157.130:55083)
0925 02:46:41.347230 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0925 02:46:41.347289 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:41669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":77,"threads_started":1}
I20260925 02:46:41.519949 31096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:41.519547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:41669 (local address 127.13.157.131:37883)
0925 02:46:41.519787 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0925 02:46:41.519849 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:41669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":67,"threads_started":1}
I20260925 02:46:41.806988 31097 raft_consensus.cc:482] T d70e788499bd47c29a5795583b21ca32 P 4b0601a8350a44b9a3e871bdf175281b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 02:46:41.807183 31097 raft_consensus.cc:504] T d70e788499bd47c29a5795583b21ca32 P 4b0601a8350a44b9a3e871bdf175281b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "519a5d55415b4a5aa62e9ada1250a9ec" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 44627 } } peers { permanent_uuid: "0fb2fcd1a5744120ae6146cbc623af16" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 45427 } } peers { permanent_uuid: "4b0601a8350a44b9a3e871bdf175281b" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 44449 } }
I20260925 02:46:41.808297 31097 leader_election.cc:288] T d70e788499bd47c29a5795583b21ca32 P 4b0601a8350a44b9a3e871bdf175281b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 519a5d55415b4a5aa62e9ada1250a9ec (127.13.157.131:44627), 0fb2fcd1a5744120ae6146cbc623af16 (127.13.157.130:45427)
Sep 25 02:46:41 dist-test-slave-kznw krb5kdc[29489](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304400, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.129@KRBTEST.COM for kudu/127.13.157.131@KRBTEST.COM
Sep 25 02:46:41 dist-test-slave-kznw krb5kdc[29489](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304400, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.129@KRBTEST.COM for kudu/127.13.157.130@KRBTEST.COM
I20260925 02:46:41.825546 31100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:41.808795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.129:54977 (local address 127.13.157.131:44627)
0925 02:46:41.809226 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0925 02:46:41.809246 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0925 02:46:41.809288 (+ 42us) server_negotiation.cc:407] Connection header received
0925 02:46:41.810230 (+ 942us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:46:41.810241 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:46:41.810712 (+ 471us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:46:41.811085 (+ 373us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:46:41.813013 (+ 1928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:46:41.814591 (+ 1578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:46:41.816534 (+ 1943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:46:41.817282 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:46:41.817463 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:46:41.820655 (+ 3192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:46:41.820664 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:46:41.820682 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:46:41.820721 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:46:41.822647 (+ 1926us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:46:41.823385 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:46:41.823390 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:46:41.823394 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:46:41.823521 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:46:41.823881 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:46:41.823883 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:46:41.823885 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:46:41.824384 (+ 499us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:46:41.824523 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0925 02:46:41.825098 (+ 575us) server_negotiation.cc:299] Negotiation successful
0925 02:46:41.825360 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":102,"threads_started":1}
I20260925 02:46:41.825551 31098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:41.808567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.131:44627 (local address 127.13.157.129:54977)
0925 02:46:41.809006 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0925 02:46:41.809018 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 02:46:41.809075 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:46:41.811193 (+ 2118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:46:41.811200 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:46:41.811945 (+ 745us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:46:41.812763 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:46:41.812789 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:46:41.814782 (+ 1993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:46:41.814786 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:46:41.816384 (+ 1598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:46:41.816397 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:46:41.817408 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:46:41.817412 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:46:41.817706 (+ 294us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:46:41.818472 (+ 766us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:46:41.818500 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:46:41.820370 (+ 1870us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:46:41.822775 (+ 2405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:46:41.822781 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:46:41.822794 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:46:41.823270 (+ 476us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:46:41.823616 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:46:41.823619 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:46:41.823621 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:46:41.823783 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:46:41.824576 (+ 793us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:46:41.824578 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:46:41.824746 (+ 168us) client_negotiation.cc:769] Sending connection context
0925 02:46:41.824942 (+ 196us) client_negotiation.cc:240] Negotiation successful
0925 02:46:41.825339 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":387,"mutex_wait_us":371,"spinlock_wait_cycles":44416,"thread_start_us":49,"threads_started":1}
I20260925 02:46:41.825549 31101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:41.808769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.129:54057 (local address 127.13.157.130:45427)
0925 02:46:41.809226 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0925 02:46:41.809243 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0925 02:46:41.809288 (+ 45us) server_negotiation.cc:407] Connection header received
0925 02:46:41.810236 (+ 948us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:46:41.810244 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:46:41.810712 (+ 468us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:46:41.811085 (+ 373us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:46:41.812990 (+ 1905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:46:41.814614 (+ 1624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:46:41.816534 (+ 1920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:46:41.817282 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:46:41.817572 (+ 290us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:46:41.820966 (+ 3394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:46:41.820971 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:46:41.820983 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:46:41.821014 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:46:41.822642 (+ 1628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:46:41.823386 (+ 744us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:46:41.823391 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:46:41.823394 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:46:41.823516 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:46:41.823874 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:46:41.823877 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:46:41.823878 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:46:41.824370 (+ 492us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:46:41.824497 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0925 02:46:41.825092 (+ 595us) server_negotiation.cc:299] Negotiation successful
0925 02:46:41.825362 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":344,"thread_start_us":120,"threads_started":1}
I20260925 02:46:41.826117 31099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:41.808567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.130:45427 (local address 127.13.157.129:54057)
0925 02:46:41.808908 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0925 02:46:41.808925 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 02:46:41.809061 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:46:41.811193 (+ 2132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:46:41.811200 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:46:41.811945 (+ 745us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:46:41.812764 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:46:41.812789 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:46:41.814783 (+ 1994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:46:41.814786 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:46:41.816384 (+ 1598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:46:41.816397 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:46:41.817408 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:46:41.817412 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:46:41.817708 (+ 296us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:46:41.818477 (+ 769us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:46:41.818500 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:46:41.820763 (+ 2263us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:46:41.822772 (+ 2009us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:46:41.822779 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:46:41.822794 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:46:41.823275 (+ 481us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:46:41.823610 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:46:41.823613 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:46:41.823615 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:46:41.823774 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:46:41.824489 (+ 715us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:46:41.824495 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:46:41.824750 (+ 255us) client_negotiation.cc:769] Sending connection context
0925 02:46:41.824943 (+ 193us) client_negotiation.cc:240] Negotiation successful
0925 02:46:41.825417 (+ 474us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"mutex_wait_us":464,"spinlock_wait_cycles":30208,"thread_start_us":79,"threads_started":1}
W20260925 02:46:41.828140 30704 leader_election.cc:341] T d70e788499bd47c29a5795583b21ca32 P 4b0601a8350a44b9a3e871bdf175281b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 519a5d55415b4a5aa62e9ada1250a9ec (127.13.157.131:44627): Not found: Tablet not found: d70e788499bd47c29a5795583b21ca32
W20260925 02:46:41.828306 30705 leader_election.cc:341] T d70e788499bd47c29a5795583b21ca32 P 4b0601a8350a44b9a3e871bdf175281b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0fb2fcd1a5744120ae6146cbc623af16 (127.13.157.130:45427): Not found: Tablet not found: d70e788499bd47c29a5795583b21ca32
I20260925 02:46:41.828379 30705 leader_election.cc:302] T d70e788499bd47c29a5795583b21ca32 P 4b0601a8350a44b9a3e871bdf175281b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b0601a8350a44b9a3e871bdf175281b; no voters: 0fb2fcd1a5744120ae6146cbc623af16, 519a5d55415b4a5aa62e9ada1250a9ec
I20260925 02:46:41.828614 31097 raft_consensus.cc:2726] T d70e788499bd47c29a5795583b21ca32 P 4b0601a8350a44b9a3e871bdf175281b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 02:46:42.201388 31099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:42.201088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:41669 (local address 127.13.157.129:57943)
0925 02:46:42.201238 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0925 02:46:42.201299 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:41669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260925 02:46:42.348573 31102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:42.348131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:41669 (local address 127.13.157.130:45353)
0925 02:46:42.348405 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0925 02:46:42.348475 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:41669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":74,"threads_started":1}
I20260925 02:46:42.521083 31103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:42.520632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:41669 (local address 127.13.157.131:55519)
0925 02:46:42.520919 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0925 02:46:42.520980 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:41669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":75,"threads_started":1}
I20260925 02:46:43.202589 31104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:43.202154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:41669 (local address 127.13.157.129:34431)
0925 02:46:43.202424 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0925 02:46:43.202485 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:41669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":78,"threads_started":1}
I20260925 02:46:43.349848 31105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:43.349400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:41669 (local address 127.13.157.130:33595)
0925 02:46:43.349686 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0925 02:46:43.349750 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:41669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":75,"threads_started":1}
I20260925 02:46:43.522308 31106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:43.521859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:41669 (local address 127.13.157.131:42989)
0925 02:46:43.522147 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0925 02:46:43.522213 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:41669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":80,"threads_started":1}
I20260925 02:46:43.883049 31107 raft_consensus.cc:482] T d70e788499bd47c29a5795583b21ca32 P 4b0601a8350a44b9a3e871bdf175281b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 02:46:43.883183 31107 raft_consensus.cc:504] T d70e788499bd47c29a5795583b21ca32 P 4b0601a8350a44b9a3e871bdf175281b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "519a5d55415b4a5aa62e9ada1250a9ec" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 44627 } } peers { permanent_uuid: "0fb2fcd1a5744120ae6146cbc623af16" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 45427 } } peers { permanent_uuid: "4b0601a8350a44b9a3e871bdf175281b" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 44449 } }
I20260925 02:46:43.883584 31107 leader_election.cc:288] T d70e788499bd47c29a5795583b21ca32 P 4b0601a8350a44b9a3e871bdf175281b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 519a5d55415b4a5aa62e9ada1250a9ec (127.13.157.131:44627), 0fb2fcd1a5744120ae6146cbc623af16 (127.13.157.130:45427)
W20260925 02:46:43.884434 30704 leader_election.cc:341] T d70e788499bd47c29a5795583b21ca32 P 4b0601a8350a44b9a3e871bdf175281b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 519a5d55415b4a5aa62e9ada1250a9ec (127.13.157.131:44627): Not found: Tablet not found: d70e788499bd47c29a5795583b21ca32
W20260925 02:46:43.884562 30705 leader_election.cc:341] T d70e788499bd47c29a5795583b21ca32 P 4b0601a8350a44b9a3e871bdf175281b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0fb2fcd1a5744120ae6146cbc623af16 (127.13.157.130:45427): Not found: Tablet not found: d70e788499bd47c29a5795583b21ca32
I20260925 02:46:43.884646 30705 leader_election.cc:302] T d70e788499bd47c29a5795583b21ca32 P 4b0601a8350a44b9a3e871bdf175281b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b0601a8350a44b9a3e871bdf175281b; no voters: 0fb2fcd1a5744120ae6146cbc623af16, 519a5d55415b4a5aa62e9ada1250a9ec
I20260925 02:46:43.884783 31107 raft_consensus.cc:2726] T d70e788499bd47c29a5795583b21ca32 P 4b0601a8350a44b9a3e871bdf175281b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 02:46:44.203797 31108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:44.203348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:41669 (local address 127.13.157.129:47027)
0925 02:46:44.203644 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0925 02:46:44.203700 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:41669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":71,"threads_started":1}
I20260925 02:46:44.351034 31109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:44.350555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:41669 (local address 127.13.157.130:49723)
0925 02:46:44.350842 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0925 02:46:44.350935 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:41669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":94,"threads_started":1}
I20260925 02:46:44.523556 31110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:44.523059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:41669 (local address 127.13.157.131:59833)
0925 02:46:44.523375 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0925 02:46:44.523448 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:41669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":95,"threads_started":1}
I20260925 02:46:45.205477 31111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:45.204887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:41669 (local address 127.13.157.129:56961)
0925 02:46:45.205274 (+ 387us) negotiation.cc:107] Waiting for socket to connect
0925 02:46:45.205342 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:41669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":113,"threads_started":1}
I20260925 02:46:45.352190 31112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:45.351735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:41669 (local address 127.13.157.130:59365)
0925 02:46:45.352029 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0925 02:46:45.352092 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:41669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":88,"threads_started":1}
I20260925 02:46:45.524789 31113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:45.524344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:41669 (local address 127.13.157.131:60345)
0925 02:46:45.524630 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0925 02:46:45.524688 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:41669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":86,"threads_started":1}
I20260925 02:46:46.206732 31114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:46.206218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:41669 (local address 127.13.157.129:47209)
0925 02:46:46.206545 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0925 02:46:46.206626 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:41669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":74,"threads_started":1}
I20260925 02:46:46.353499 31115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:46.353002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:41669 (local address 127.13.157.130:35135)
0925 02:46:46.353291 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0925 02:46:46.353355 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:41669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":95,"threads_started":1}
I20260925 02:46:46.525961 31116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:46.525565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:41669 (local address 127.13.157.131:38821)
0925 02:46:46.525804 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0925 02:46:46.525864 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:41669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":74,"threads_started":1}
I20260925 02:46:46.707033 31117 raft_consensus.cc:482] T d70e788499bd47c29a5795583b21ca32 P 4b0601a8350a44b9a3e871bdf175281b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 02:46:46.707165 31117 raft_consensus.cc:504] T d70e788499bd47c29a5795583b21ca32 P 4b0601a8350a44b9a3e871bdf175281b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "519a5d55415b4a5aa62e9ada1250a9ec" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 44627 } } peers { permanent_uuid: "0fb2fcd1a5744120ae6146cbc623af16" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 45427 } } peers { permanent_uuid: "4b0601a8350a44b9a3e871bdf175281b" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 44449 } }
I20260925 02:46:46.707628 31117 leader_election.cc:288] T d70e788499bd47c29a5795583b21ca32 P 4b0601a8350a44b9a3e871bdf175281b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 519a5d55415b4a5aa62e9ada1250a9ec (127.13.157.131:44627), 0fb2fcd1a5744120ae6146cbc623af16 (127.13.157.130:45427)
W20260925 02:46:46.708392 30704 leader_election.cc:341] T d70e788499bd47c29a5795583b21ca32 P 4b0601a8350a44b9a3e871bdf175281b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 519a5d55415b4a5aa62e9ada1250a9ec (127.13.157.131:44627): Not found: Tablet not found: d70e788499bd47c29a5795583b21ca32
W20260925 02:46:46.708552 30705 leader_election.cc:341] T d70e788499bd47c29a5795583b21ca32 P 4b0601a8350a44b9a3e871bdf175281b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0fb2fcd1a5744120ae6146cbc623af16 (127.13.157.130:45427): Not found: Tablet not found: d70e788499bd47c29a5795583b21ca32
I20260925 02:46:46.708617 30705 leader_election.cc:302] T d70e788499bd47c29a5795583b21ca32 P 4b0601a8350a44b9a3e871bdf175281b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b0601a8350a44b9a3e871bdf175281b; no voters: 0fb2fcd1a5744120ae6146cbc623af16, 519a5d55415b4a5aa62e9ada1250a9ec
I20260925 02:46:46.708752 31117 raft_consensus.cc:2726] T d70e788499bd47c29a5795583b21ca32 P 4b0601a8350a44b9a3e871bdf175281b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 02:46:47.207895 31118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:47.207475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:41669 (local address 127.13.157.129:42989)
0925 02:46:47.207741 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0925 02:46:47.207795 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:41669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":80,"threads_started":1}
I20260925 02:46:47.354820 31119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:47.354284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:41669 (local address 127.13.157.130:36087)
0925 02:46:47.354658 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0925 02:46:47.354720 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:41669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":71,"threads_started":1}
I20260925 02:46:47.527189 31120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:47.526696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:41669 (local address 127.13.157.131:49269)
0925 02:46:47.527034 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0925 02:46:47.527096 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:41669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":72,"threads_started":1}
I20260925 02:46:48.209079 31121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:48.208597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:41669 (local address 127.13.157.129:55271)
0925 02:46:48.208880 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0925 02:46:48.208943 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:41669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":86,"threads_started":1}
I20260925 02:46:48.356204 31122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:48.355688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:41669 (local address 127.13.157.130:49395)
0925 02:46:48.356017 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0925 02:46:48.356090 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:41669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":96,"threads_started":1}
I20260925 02:46:48.528420 31123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:48.527932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:41669 (local address 127.13.157.131:53105)
0925 02:46:48.528231 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0925 02:46:48.528318 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:41669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":78,"threads_started":1}
I20260925 02:46:48.891779 31124 raft_consensus.cc:482] T d70e788499bd47c29a5795583b21ca32 P 4b0601a8350a44b9a3e871bdf175281b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 02:46:48.891918 31124 raft_consensus.cc:504] T d70e788499bd47c29a5795583b21ca32 P 4b0601a8350a44b9a3e871bdf175281b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "519a5d55415b4a5aa62e9ada1250a9ec" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 44627 } } peers { permanent_uuid: "0fb2fcd1a5744120ae6146cbc623af16" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 45427 } } peers { permanent_uuid: "4b0601a8350a44b9a3e871bdf175281b" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 44449 } }
I20260925 02:46:48.892391 31124 leader_election.cc:288] T d70e788499bd47c29a5795583b21ca32 P 4b0601a8350a44b9a3e871bdf175281b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 519a5d55415b4a5aa62e9ada1250a9ec (127.13.157.131:44627), 0fb2fcd1a5744120ae6146cbc623af16 (127.13.157.130:45427)
W20260925 02:46:48.893150 30704 leader_election.cc:341] T d70e788499bd47c29a5795583b21ca32 P 4b0601a8350a44b9a3e871bdf175281b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 519a5d55415b4a5aa62e9ada1250a9ec (127.13.157.131:44627): Not found: Tablet not found: d70e788499bd47c29a5795583b21ca32
W20260925 02:46:48.893260 30705 leader_election.cc:341] T d70e788499bd47c29a5795583b21ca32 P 4b0601a8350a44b9a3e871bdf175281b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0fb2fcd1a5744120ae6146cbc623af16 (127.13.157.130:45427): Not found: Tablet not found: d70e788499bd47c29a5795583b21ca32
I20260925 02:46:48.893378 30705 leader_election.cc:302] T d70e788499bd47c29a5795583b21ca32 P 4b0601a8350a44b9a3e871bdf175281b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b0601a8350a44b9a3e871bdf175281b; no voters: 0fb2fcd1a5744120ae6146cbc623af16, 519a5d55415b4a5aa62e9ada1250a9ec
I20260925 02:46:48.893539 31124 raft_consensus.cc:2726] T d70e788499bd47c29a5795583b21ca32 P 4b0601a8350a44b9a3e871bdf175281b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 02:46:49.210352 31125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:49.209887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:41669 (local address 127.13.157.129:57735)
0925 02:46:49.210177 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0925 02:46:49.210235 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:41669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":92,"threads_started":1}
I20260925 02:46:49.357422 31126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:49.356953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:41669 (local address 127.13.157.130:36175)
0925 02:46:49.357261 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0925 02:46:49.357324 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:41669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":108,"threads_started":1}
I20260925 02:46:49.529553 31127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:49.529133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:41669 (local address 127.13.157.131:42975)
0925 02:46:49.529402 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0925 02:46:49.529459 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:41669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":77,"threads_started":1}
I20260925 02:46:50.211760 31128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:50.211277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:41669 (local address 127.13.157.129:42699)
0925 02:46:50.211587 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0925 02:46:50.211655 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:41669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":71,"threads_started":1}
I20260925 02:46:50.358834 31129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:50.358291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:41669 (local address 127.13.157.130:43077)
0925 02:46:50.358642 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0925 02:46:50.358715 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:41669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":93,"threads_started":1}
I20260925 02:46:50.530764 31130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:50.530301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:41669 (local address 127.13.157.131:36753)
0925 02:46:50.530607 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0925 02:46:50.530667 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:41669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":90,"threads_started":1}
I20260925 02:46:51.101573 31131 raft_consensus.cc:482] T d70e788499bd47c29a5795583b21ca32 P 4b0601a8350a44b9a3e871bdf175281b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 02:46:51.101774 31131 raft_consensus.cc:504] T d70e788499bd47c29a5795583b21ca32 P 4b0601a8350a44b9a3e871bdf175281b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "519a5d55415b4a5aa62e9ada1250a9ec" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 44627 } } peers { permanent_uuid: "0fb2fcd1a5744120ae6146cbc623af16" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 45427 } } peers { permanent_uuid: "4b0601a8350a44b9a3e871bdf175281b" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 44449 } }
I20260925 02:46:51.102226 31131 leader_election.cc:288] T d70e788499bd47c29a5795583b21ca32 P 4b0601a8350a44b9a3e871bdf175281b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 519a5d55415b4a5aa62e9ada1250a9ec (127.13.157.131:44627), 0fb2fcd1a5744120ae6146cbc623af16 (127.13.157.130:45427)
W20260925 02:46:51.102998 30705 leader_election.cc:341] T d70e788499bd47c29a5795583b21ca32 P 4b0601a8350a44b9a3e871bdf175281b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0fb2fcd1a5744120ae6146cbc623af16 (127.13.157.130:45427): Not found: Tablet not found: d70e788499bd47c29a5795583b21ca32
W20260925 02:46:51.103099 30704 leader_election.cc:341] T d70e788499bd47c29a5795583b21ca32 P 4b0601a8350a44b9a3e871bdf175281b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 519a5d55415b4a5aa62e9ada1250a9ec (127.13.157.131:44627): Not found: Tablet not found: d70e788499bd47c29a5795583b21ca32
I20260925 02:46:51.103204 30704 leader_election.cc:302] T d70e788499bd47c29a5795583b21ca32 P 4b0601a8350a44b9a3e871bdf175281b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b0601a8350a44b9a3e871bdf175281b; no voters: 0fb2fcd1a5744120ae6146cbc623af16, 519a5d55415b4a5aa62e9ada1250a9ec
I20260925 02:46:51.103363 31131 raft_consensus.cc:2726] T d70e788499bd47c29a5795583b21ca32 P 4b0601a8350a44b9a3e871bdf175281b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 02:46:51.212904 31132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:51.212457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:41669 (local address 127.13.157.129:39963)
0925 02:46:51.212743 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0925 02:46:51.212799 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:41669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":89,"threads_started":1}
I20260925 02:46:51.360047 31133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:51.359607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:41669 (local address 127.13.157.130:42539)
0925 02:46:51.359881 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0925 02:46:51.359945 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:41669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":89,"threads_started":1}
I20260925 02:46:51.531922 31134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:51.531490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:41669 (local address 127.13.157.131:55381)
0925 02:46:51.531763 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0925 02:46:51.531823 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:41669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":74,"threads_started":1}
I20260925 02:46:52.214157 31135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:52.213678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:41669 (local address 127.13.157.129:41437)
0925 02:46:52.213978 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0925 02:46:52.214053 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:41669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":98,"threads_started":1}
I20260925 02:46:52.361307 31136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:52.360859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:41669 (local address 127.13.157.130:49671)
0925 02:46:52.361146 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0925 02:46:52.361207 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:41669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":75,"threads_started":1}
I20260925 02:46:52.532968 31137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:52.532545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:41669 (local address 127.13.157.131:56791)
0925 02:46:52.532812 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0925 02:46:52.532871 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:41669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":84,"threads_started":1}
I20260925 02:46:53.215389 31138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:53.214978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:41669 (local address 127.13.157.129:35301)
0925 02:46:53.215224 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0925 02:46:53.215290 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:41669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":73,"threads_started":1}
I20260925 02:46:53.362574 31139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:53.362121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:41669 (local address 127.13.157.130:37995)
0925 02:46:53.362397 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0925 02:46:53.362461 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:41669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":75,"threads_started":1}
I20260925 02:46:53.534183 31140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:53.533698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:41669 (local address 127.13.157.131:37959)
0925 02:46:53.533994 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0925 02:46:53.534066 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:41669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":86,"threads_started":1}
I20260925 02:46:54.216574 31141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:54.216145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:41669 (local address 127.13.157.129:48925)
0925 02:46:54.216401 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0925 02:46:54.216457 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:41669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":80,"threads_started":1}
I20260925 02:46:54.363817 31142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:54.363336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:41669 (local address 127.13.157.130:47663)
0925 02:46:54.363652 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0925 02:46:54.363718 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:41669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":91,"threads_started":1}
I20260925 02:46:54.535531 31143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:54.535019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:41669 (local address 127.13.157.131:34547)
0925 02:46:54.535334 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0925 02:46:54.535411 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:41669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":83,"threads_started":1}
I20260925 02:46:55.217949 31144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:55.217504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:41669 (local address 127.13.157.129:34623)
0925 02:46:55.217783 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0925 02:46:55.217835 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:41669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":113,"threads_started":1}
I20260925 02:46:55.365247 31145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:55.364784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:41669 (local address 127.13.157.130:35195)
0925 02:46:55.365081 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0925 02:46:55.365146 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.190:41669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":75,"threads_started":1}
I20260925 02:46:55.522048 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 30648
I20260925 02:46:55.527812 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 30823
I20260925 02:46:55.532763 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 30960
2026-09-25T02:46:55Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (59157 ms)
[----------] 30 tests from SecurityITest (237052 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790304178325258-13942-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 02:46:55 dist-test-slave-kznw krb5kdc[31148](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 02:46:55 dist-test-slave-kznw krb5kdc[31148](info): set up 2 sockets
Sep 25 02:46:55 dist-test-slave-kznw krb5kdc[31148](info): commencing operation
krb5kdc: starting...
W20260925 02:46:56.498298 13942 mini_kdc.cc:121] Time spent starting KDC: real 0.945s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790304178325258-13942-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790304178325258-13942-0/krb5kdc/test-user.keytab.
Sep 25 02:46:56 dist-test-slave-kznw krb5kdc[31148](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304416, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T02:46:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T02:46:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.157.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.157.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.157.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab.
Entry for principal kudu/127.13.157.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab.
I20260925 02:46:56.618119 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790304178325258-13942-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790304178325258-13942-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790304178325258-13942-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.157.190:42857
--webserver_interface=127.13.157.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab
--principal=kudu/127.13.157.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:38915
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.157.190:42857
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:46:56.728909 31161 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:46:56.729220 31161 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:46:56.729282 31161 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:46:56.733332 31161 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:46:56.733397 31161 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:46:56.733433 31161 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:46:56.733455 31161 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:46:56.733484 31161 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:46:56.738878 31161 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:38915
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790304178325258-13942-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790304178325258-13942-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.157.190:42857
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab
--principal=kudu/127.13.157.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.157.190:42857
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.157.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31161
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790304178325258-13942-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:46:56.740020 31161 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:46:56.741043 31161 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 02:46:56.749821 31161 server_base.cc:1034] running on GCE node
W20260925 02:46:56.749764 31167 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:46:56.749778 31166 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:46:56.749771 31169 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:46:56.750535 31161 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:46:56.751679 31161 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:46:56.752844 31161 hybrid_clock.cc:648] HybridClock initialized: now 1790304416752822 us; error 33 us; skew 500 ppm
Sep 25 02:46:56 dist-test-slave-kznw krb5kdc[31148](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304416, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:46:56.755601 31161 init.cc:377] Logged in from keytab as kudu/127.13.157.190@KRBTEST.COM (short username kudu)
I20260925 02:46:56.756449 31161 webserver.cc:460] Webserver started at http://127.13.157.190:41757/ using document root <none> and password file <none>
I20260925 02:46:56.756994 31161 fs_manager.cc:359] Metadata directory not provided
I20260925 02:46:56.757058 31161 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:46:56.757251 31161 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:46:56.759377 31161 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790304178325258-13942-0/minicluster-data/master-0/data/instance:
uuid: "63515883fb3141769fde9dbae17dd934"
format_stamp: "Formatted at 2026-09-25 02:46:56 on dist-test-slave-kznw"
server_key: "ba120909ab41faae7ab2d21410921127"
server_key_iv: "1375ef7ae97733948ec0751228563d08"
server_key_version: "encryptionkey@0"
I20260925 02:46:56.759943 31161 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790304178325258-13942-0/minicluster-data/master-0/wal/instance:
uuid: "63515883fb3141769fde9dbae17dd934"
format_stamp: "Formatted at 2026-09-25 02:46:56 on dist-test-slave-kznw"
server_key: "ba120909ab41faae7ab2d21410921127"
server_key_iv: "1375ef7ae97733948ec0751228563d08"
server_key_version: "encryptionkey@0"
I20260925 02:46:56.763798 31161 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260925 02:46:56.766242 31176 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:46:56.767587 31161 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260925 02:46:56.767709 31161 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790304178325258-13942-0/minicluster-data/master-0/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790304178325258-13942-0/minicluster-data/master-0/wal
uuid: "63515883fb3141769fde9dbae17dd934"
format_stamp: "Formatted at 2026-09-25 02:46:56 on dist-test-slave-kznw"
server_key: "ba120909ab41faae7ab2d21410921127"
server_key_iv: "1375ef7ae97733948ec0751228563d08"
server_key_version: "encryptionkey@0"
I20260925 02:46:56.767812 31161 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790304178325258-13942-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790304178325258-13942-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790304178325258-13942-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:46:56.779752 31161 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:46:56.782800 31161 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:46:56.783033 31161 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:46:56.789721 31161 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.190:42857
I20260925 02:46:56.789758 31228 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.190:42857 every 8 connection(s)
I20260925 02:46:56.790653 31161 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
I20260925 02:46:56.793447 31229 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:46:56.794586 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 31161
I20260925 02:46:56.794732 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790304178325258-13942-0/minicluster-data/master-0/wal/instance
I20260925 02:46:56.795082 13942 external_mini_cluster.cc:1428] Setting key 90382323816bd0845098f83e3ab83b0d
I20260925 02:46:56.799499 31229 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63515883fb3141769fde9dbae17dd934: Bootstrap starting.
I20260925 02:46:56.801684 31229 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 63515883fb3141769fde9dbae17dd934: Neither blocks nor log segments found. Creating new log.
I20260925 02:46:56.802481 31229 log.cc:824] T 00000000000000000000000000000000 P 63515883fb3141769fde9dbae17dd934: Log is configured to *not* fsync() on all Append() calls
Sep 25 02:46:56 dist-test-slave-kznw krb5kdc[31148](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304416, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.157.190@KRBTEST.COM
I20260925 02:46:56.804374 31229 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63515883fb3141769fde9dbae17dd934: No bootstrap required, opened a new log
I20260925 02:46:56.808305 31232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:56.796544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32786 (local address 127.13.157.190:42857)
0925 02:46:56.796970 (+ 426us) server_negotiation.cc:206] Beginning negotiation
0925 02:46:56.796985 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0925 02:46:56.797020 (+ 35us) server_negotiation.cc:407] Connection header received
0925 02:46:56.797657 (+ 637us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:46:56.797673 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:46:56.797987 (+ 314us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:46:56.798292 (+ 305us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:46:56.799115 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:46:56.800128 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:46:56.800970 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:46:56.801435 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:46:56.801584 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:46:56.803981 (+ 2397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:46:56.803986 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:46:56.803999 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:46:56.804025 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:46:56.805619 (+ 1594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:46:56.806086 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:46:56.806090 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:46:56.806094 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:46:56.806197 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:46:56.806455 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:46:56.806457 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:46:56.806458 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:46:56.806852 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:46:56.807029 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0925 02:46:56.807291 (+ 262us) server_negotiation.cc:299] Negotiation successful
0925 02:46:56.807584 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":281,"thread_start_us":119,"threads_started":1}
I20260925 02:46:56.808267 31229 raft_consensus.cc:348] T 00000000000000000000000000000000 P 63515883fb3141769fde9dbae17dd934 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63515883fb3141769fde9dbae17dd934" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 42857 } }
I20260925 02:46:56.808511 31229 raft_consensus.cc:374] T 00000000000000000000000000000000 P 63515883fb3141769fde9dbae17dd934 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:46:56.808573 31229 raft_consensus.cc:729] T 00000000000000000000000000000000 P 63515883fb3141769fde9dbae17dd934 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63515883fb3141769fde9dbae17dd934, State: Initialized, Role: FOLLOWER
I20260925 02:46:56.808997 31229 consensus_queue.cc:260] T 00000000000000000000000000000000 P 63515883fb3141769fde9dbae17dd934 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63515883fb3141769fde9dbae17dd934" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 42857 } }
I20260925 02:46:56.809140 31229 raft_consensus.cc:388] T 00000000000000000000000000000000 P 63515883fb3141769fde9dbae17dd934 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 02:46:56.810055 31229 raft_consensus.cc:482] T 00000000000000000000000000000000 P 63515883fb3141769fde9dbae17dd934 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 02:46:56.810175 31229 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 63515883fb3141769fde9dbae17dd934 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:46:56.811048 31229 raft_consensus.cc:504] T 00000000000000000000000000000000 P 63515883fb3141769fde9dbae17dd934 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63515883fb3141769fde9dbae17dd934" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 42857 } }
I20260925 02:46:56.811371 31229 leader_election.cc:302] T 00000000000000000000000000000000 P 63515883fb3141769fde9dbae17dd934 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 63515883fb3141769fde9dbae17dd934; no voters:
I20260925 02:46:56.811646 31229 leader_election.cc:288] T 00000000000000000000000000000000 P 63515883fb3141769fde9dbae17dd934 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 02:46:56.811753 31234 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 63515883fb3141769fde9dbae17dd934 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:46:56.811944 31234 raft_consensus.cc:686] T 00000000000000000000000000000000 P 63515883fb3141769fde9dbae17dd934 [term 1 LEADER]: Becoming Leader. State: Replica: 63515883fb3141769fde9dbae17dd934, State: Running, Role: LEADER
I20260925 02:46:56.812290 31234 consensus_queue.cc:237] T 00000000000000000000000000000000 P 63515883fb3141769fde9dbae17dd934 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63515883fb3141769fde9dbae17dd934" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 42857 } }
I20260925 02:46:56.812813 31229 sys_catalog.cc:564] T 00000000000000000000000000000000 P 63515883fb3141769fde9dbae17dd934 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:46:56.813921 31235 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63515883fb3141769fde9dbae17dd934 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "63515883fb3141769fde9dbae17dd934" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63515883fb3141769fde9dbae17dd934" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 42857 } } }
I20260925 02:46:56.814050 31235 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63515883fb3141769fde9dbae17dd934 [sys.catalog]: This master's current role is: LEADER
I20260925 02:46:56.814422 31243 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:46:56.814400 31236 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63515883fb3141769fde9dbae17dd934 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 63515883fb3141769fde9dbae17dd934. Latest consensus state: current_term: 1 leader_uuid: "63515883fb3141769fde9dbae17dd934" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63515883fb3141769fde9dbae17dd934" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 42857 } } }
I20260925 02:46:56.815086 31236 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63515883fb3141769fde9dbae17dd934 [sys.catalog]: This master's current role is: LEADER
I20260925 02:46:56.817087 31243 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:46:56.822733 31243 catalog_manager.cc:1347] Generated new cluster ID: f2706b36938f475bb4a1ba8da413876e
I20260925 02:46:56.822809 31243 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 02:46:56.853161 31243 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 02:46:56.854230 31243 catalog_manager.cc:1504] Loading token signing keys...
I20260925 02:46:56.870144 31243 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 63515883fb3141769fde9dbae17dd934: Generated new TSK 0
I20260925 02:46:56.870867 31243 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.157.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.157.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.157.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.129.keytab.
Entry for principal kudu/127.13.157.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.129.keytab.
I20260925 02:46:56.906736 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790304178325258-13942-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790304178325258-13942-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790304178325258-13942-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790304178325258-13942-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.157.129:0
--local_ip_for_outbound_sockets=127.13.157.129
--webserver_interface=127.13.157.129
--webserver_port=0
--tserver_master_addrs=127.13.157.190:42857
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.129.keytab
--principal=kudu/127.13.157.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:38915
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:46:57.015939 31255 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:46:57.016196 31255 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:46:57.016253 31255 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:46:57.020210 31255 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:46:57.020269 31255 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:46:57.020347 31255 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.157.129
I20260925 02:46:57.025610 31255 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:38915
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790304178325258-13942-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790304178325258-13942-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.129.keytab
--principal=kudu/127.13.157.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.157.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790304178325258-13942-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.157.129
--webserver_port=0
--tserver_master_addrs=127.13.157.190:42857
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31255
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.157.129
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790304178325258-13942-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:46:57.026763 31255 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:46:57.027814 31255 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:46:57.037341 31260 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:46:57.037428 31255 server_base.cc:1034] running on GCE node
W20260925 02:46:57.037348 31263 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:46:57.037361 31261 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:46:57.038096 31255 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:46:57.038834 31255 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:46:57.040055 31255 hybrid_clock.cc:648] HybridClock initialized: now 1790304417040033 us; error 45 us; skew 500 ppm
Sep 25 02:46:57 dist-test-slave-kznw krb5kdc[31148](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304417, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:46:57.042946 31255 init.cc:377] Logged in from keytab as kudu/127.13.157.129@KRBTEST.COM (short username kudu)
I20260925 02:46:57.043777 31255 webserver.cc:460] Webserver started at http://127.13.157.129:42309/ using document root <none> and password file <none>
I20260925 02:46:57.044322 31255 fs_manager.cc:359] Metadata directory not provided
I20260925 02:46:57.044380 31255 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:46:57.044544 31255 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:46:57.046586 31255 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790304178325258-13942-0/minicluster-data/ts-0/data/instance:
uuid: "d59e88b0215b4834ac7f76941b56bcc6"
format_stamp: "Formatted at 2026-09-25 02:46:57 on dist-test-slave-kznw"
server_key: "bc5ebc519f6ee0ce4aba5520ac967448"
server_key_iv: "cdd8575c46ca15e55bc24b61505e5c43"
server_key_version: "encryptionkey@0"
I20260925 02:46:57.047151 31255 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790304178325258-13942-0/minicluster-data/ts-0/wal/instance:
uuid: "d59e88b0215b4834ac7f76941b56bcc6"
format_stamp: "Formatted at 2026-09-25 02:46:57 on dist-test-slave-kznw"
server_key: "bc5ebc519f6ee0ce4aba5520ac967448"
server_key_iv: "cdd8575c46ca15e55bc24b61505e5c43"
server_key_version: "encryptionkey@0"
I20260925 02:46:57.050853 31255 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260925 02:46:57.053580 31270 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:46:57.054894 31255 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260925 02:46:57.055075 31255 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790304178325258-13942-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790304178325258-13942-0/minicluster-data/ts-0/wal
uuid: "d59e88b0215b4834ac7f76941b56bcc6"
format_stamp: "Formatted at 2026-09-25 02:46:57 on dist-test-slave-kznw"
server_key: "bc5ebc519f6ee0ce4aba5520ac967448"
server_key_iv: "cdd8575c46ca15e55bc24b61505e5c43"
server_key_version: "encryptionkey@0"
I20260925 02:46:57.055183 31255 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790304178325258-13942-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790304178325258-13942-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790304178325258-13942-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:46:57.069763 31255 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:46:57.072954 31255 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:46:57.073149 31255 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:46:57.073751 31255 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:46:57.074570 31255 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:46:57.074615 31255 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:46:57.074656 31255 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:46:57.074685 31255 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:46:57.083112 31255 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.129:42795
I20260925 02:46:57.083214 31383 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.129:42795 every 8 connection(s)
I20260925 02:46:57.084015 31255 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790304178325258-13942-0/minicluster-data/ts-0/data/info.pb
I20260925 02:46:57.093466 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 31255
Sep 25 02:46:57 dist-test-slave-kznw krb5kdc[31148](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304417, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.129@KRBTEST.COM for kudu/127.13.157.190@KRBTEST.COM
I20260925 02:46:57.093674 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790304178325258-13942-0/minicluster-data/ts-0/wal/instance
I20260925 02:46:57.094020 13942 external_mini_cluster.cc:1428] Setting key 9674967bb544cae460907f0a86bc5e62
I20260925 02:46:57.098088 31232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:57.085859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.129:35047 (local address 127.13.157.190:42857)
0925 02:46:57.086059 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0925 02:46:57.086064 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:46:57.086880 (+ 816us) server_negotiation.cc:407] Connection header received
0925 02:46:57.088358 (+ 1478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:46:57.088361 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:46:57.088410 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:46:57.088484 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:46:57.089753 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:46:57.090280 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:46:57.091510 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:46:57.091792 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:46:57.091876 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:46:57.094320 (+ 2444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:46:57.094323 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:46:57.094325 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:46:57.094351 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:46:57.095951 (+ 1600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:46:57.096452 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:46:57.096455 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:46:57.096457 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:46:57.096555 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:46:57.096851 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:46:57.096854 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:46:57.096855 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:46:57.097075 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:46:57.097170 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0925 02:46:57.097817 (+ 647us) server_negotiation.cc:299] Negotiation successful
0925 02:46:57.097932 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260925 02:46:57.099057 31386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:57.086247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:42857 (local address 127.13.157.129:35047)
0925 02:46:57.086723 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0925 02:46:57.086768 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0925 02:46:57.088133 (+ 1365us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:46:57.088729 (+ 596us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:46:57.088737 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:46:57.089101 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:46:57.089600 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:46:57.089616 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:46:57.090416 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:46:57.090420 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:46:57.091389 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:46:57.091400 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:46:57.091926 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:46:57.091929 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:46:57.092090 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:46:57.092610 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:46:57.092625 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:46:57.094154 (+ 1529us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:46:57.096096 (+ 1942us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:46:57.096101 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:46:57.096112 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:46:57.096360 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:46:57.096640 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:46:57.096642 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:46:57.096643 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:46:57.096776 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:46:57.097173 (+ 397us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:46:57.097179 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:46:57.097503 (+ 324us) client_negotiation.cc:769] Sending connection context
0925 02:46:57.097758 (+ 255us) client_negotiation.cc:240] Negotiation successful
0925 02:46:57.098107 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":114,"threads_started":1}
I20260925 02:46:57.100397 31384 heartbeater.cc:347] Connected to a master server at 127.13.157.190:42857
I20260925 02:46:57.100682 31384 heartbeater.cc:464] Registering TS with master...
I20260925 02:46:57.101333 31384 heartbeater.cc:511] Master 127.13.157.190:42857 requested a full tablet report, sending...
I20260925 02:46:57.103114 31193 ts_manager.cc:194] Registered new tserver with Master: d59e88b0215b4834ac7f76941b56bcc6 (127.13.157.129:42795)
WARNING: no policy specified for kudu/127.13.157.130@KRBTEST.COM; defaulting to no policy
I20260925 02:46:57.104292 31193 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.157.129@KRBTEST.COM'} at 127.13.157.129:35047
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.157.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.157.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.130.keytab.
Entry for principal kudu/127.13.157.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.130.keytab.
I20260925 02:46:57.121465 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790304178325258-13942-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790304178325258-13942-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790304178325258-13942-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790304178325258-13942-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.157.130:0
--local_ip_for_outbound_sockets=127.13.157.130
--webserver_interface=127.13.157.130
--webserver_port=0
--tserver_master_addrs=127.13.157.190:42857
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.130.keytab
--principal=kudu/127.13.157.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:38915
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:46:57.232575 31389 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:46:57.232851 31389 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:46:57.232895 31389 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:46:57.237109 31389 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:46:57.237177 31389 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:46:57.237239 31389 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.157.130
I20260925 02:46:57.242549 31389 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:38915
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790304178325258-13942-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790304178325258-13942-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.130.keytab
--principal=kudu/127.13.157.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.157.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790304178325258-13942-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.157.130
--webserver_port=0
--tserver_master_addrs=127.13.157.190:42857
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31389
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.157.130
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790304178325258-13942-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:46:57.243744 31389 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:46:57.244796 31389 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 02:46:57.254611 31389 server_base.cc:1034] running on GCE node
W20260925 02:46:57.254573 31397 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:46:57.254565 31394 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:46:57.254573 31395 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:46:57.255226 31389 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:46:57.255925 31389 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:46:57.257084 31389 hybrid_clock.cc:648] HybridClock initialized: now 1790304417257067 us; error 34 us; skew 500 ppm
Sep 25 02:46:57 dist-test-slave-kznw krb5kdc[31148](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304417, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:46:57.259805 31389 init.cc:377] Logged in from keytab as kudu/127.13.157.130@KRBTEST.COM (short username kudu)
I20260925 02:46:57.260665 31389 webserver.cc:460] Webserver started at http://127.13.157.130:39217/ using document root <none> and password file <none>
I20260925 02:46:57.261205 31389 fs_manager.cc:359] Metadata directory not provided
I20260925 02:46:57.261270 31389 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:46:57.261505 31389 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:46:57.263557 31389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790304178325258-13942-0/minicluster-data/ts-1/data/instance:
uuid: "45b1e950347a4fc8b9ca963fa3841394"
format_stamp: "Formatted at 2026-09-25 02:46:57 on dist-test-slave-kznw"
server_key: "307350014cf1f7f56d415909d3172cec"
server_key_iv: "5d5eaf04cfe7fff00e83e6da484e2a4e"
server_key_version: "encryptionkey@0"
I20260925 02:46:57.264077 31389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790304178325258-13942-0/minicluster-data/ts-1/wal/instance:
uuid: "45b1e950347a4fc8b9ca963fa3841394"
format_stamp: "Formatted at 2026-09-25 02:46:57 on dist-test-slave-kznw"
server_key: "307350014cf1f7f56d415909d3172cec"
server_key_iv: "5d5eaf04cfe7fff00e83e6da484e2a4e"
server_key_version: "encryptionkey@0"
I20260925 02:46:57.267810 31389 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260925 02:46:57.270291 31404 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:46:57.271721 31389 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260925 02:46:57.271831 31389 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790304178325258-13942-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790304178325258-13942-0/minicluster-data/ts-1/wal
uuid: "45b1e950347a4fc8b9ca963fa3841394"
format_stamp: "Formatted at 2026-09-25 02:46:57 on dist-test-slave-kznw"
server_key: "307350014cf1f7f56d415909d3172cec"
server_key_iv: "5d5eaf04cfe7fff00e83e6da484e2a4e"
server_key_version: "encryptionkey@0"
I20260925 02:46:57.271929 31389 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790304178325258-13942-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790304178325258-13942-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790304178325258-13942-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:46:57.299574 31389 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:46:57.302804 31389 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:46:57.303020 31389 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:46:57.303650 31389 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:46:57.304471 31389 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:46:57.304526 31389 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:46:57.304580 31389 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:46:57.304606 31389 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:46:57.313163 31389 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.130:46235
I20260925 02:46:57.313194 31518 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.130:46235 every 8 connection(s)
I20260925 02:46:57.314126 31389 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790304178325258-13942-0/minicluster-data/ts-1/data/info.pb
I20260925 02:46:57.317723 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 31389
I20260925 02:46:57.317842 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790304178325258-13942-0/minicluster-data/ts-1/wal/instance
I20260925 02:46:57.318192 13942 external_mini_cluster.cc:1428] Setting key 1a597a2b66dbdddf476b7323f93d06c6
Sep 25 02:46:57 dist-test-slave-kznw krb5kdc[31148](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304417, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.130@KRBTEST.COM for kudu/127.13.157.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.157.131@KRBTEST.COM; defaulting to no policy
I20260925 02:46:57.328352 31232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:57.315880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.130:54007 (local address 127.13.157.190:42857)
0925 02:46:57.316002 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0925 02:46:57.316005 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 02:46:57.316889 (+ 884us) server_negotiation.cc:407] Connection header received
0925 02:46:57.318394 (+ 1505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:46:57.318397 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:46:57.318447 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:46:57.318529 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:46:57.320016 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:46:57.320645 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:46:57.322052 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:46:57.322276 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:46:57.322366 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:46:57.324824 (+ 2458us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:46:57.324827 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:46:57.324830 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:46:57.324855 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:46:57.326181 (+ 1326us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:46:57.326774 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:46:57.326776 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:46:57.326778 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:46:57.326852 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:46:57.327234 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:46:57.327236 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:46:57.327237 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:46:57.327430 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:46:57.327506 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0925 02:46:57.328036 (+ 530us) server_negotiation.cc:299] Negotiation successful
0925 02:46:57.328192 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260925 02:46:57.329149 31521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:57.316267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:42857 (local address 127.13.157.130:54007)
0925 02:46:57.316733 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0925 02:46:57.316776 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0925 02:46:57.318120 (+ 1344us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:46:57.318742 (+ 622us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:46:57.318750 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:46:57.319238 (+ 488us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:46:57.319837 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:46:57.319857 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:46:57.320783 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:46:57.320786 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:46:57.321926 (+ 1140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:46:57.321936 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:46:57.322419 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:46:57.322423 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:46:57.322599 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:46:57.323177 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:46:57.323194 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:46:57.324651 (+ 1457us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:46:57.326300 (+ 1649us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:46:57.326307 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:46:57.326324 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:46:57.326677 (+ 353us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:46:57.326958 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:46:57.326962 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:46:57.326964 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:46:57.327151 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:46:57.327530 (+ 379us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:46:57.327537 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:46:57.327803 (+ 266us) client_negotiation.cc:769] Sending connection context
0925 02:46:57.327960 (+ 157us) client_negotiation.cc:240] Negotiation successful
0925 02:46:57.328250 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":114,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.157.131@KRBTEST.COM" created.
I20260925 02:46:57.330366 31519 heartbeater.cc:347] Connected to a master server at 127.13.157.190:42857
I20260925 02:46:57.330662 31519 heartbeater.cc:464] Registering TS with master...
I20260925 02:46:57.331259 31519 heartbeater.cc:511] Master 127.13.157.190:42857 requested a full tablet report, sending...
I20260925 02:46:57.332360 31193 ts_manager.cc:194] Registered new tserver with Master: 45b1e950347a4fc8b9ca963fa3841394 (127.13.157.130:46235)
I20260925 02:46:57.332944 31193 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.157.130@KRBTEST.COM'} at 127.13.157.130:54007
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.157.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.131.keytab.
Entry for principal kudu/127.13.157.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.131.keytab.
I20260925 02:46:57.347062 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790304178325258-13942-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790304178325258-13942-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790304178325258-13942-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790304178325258-13942-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.157.131:0
--local_ip_for_outbound_sockets=127.13.157.131
--webserver_interface=127.13.157.131
--webserver_port=0
--tserver_master_addrs=127.13.157.190:42857
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.131.keytab
--principal=kudu/127.13.157.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:38915
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:46:57.454365 31524 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:46:57.454619 31524 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:46:57.454679 31524 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:46:57.458624 31524 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:46:57.458675 31524 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:46:57.458752 31524 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.157.131
I20260925 02:46:57.463967 31524 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:38915
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790304178325258-13942-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790304178325258-13942-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.131.keytab
--principal=kudu/127.13.157.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.157.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790304178325258-13942-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.157.131
--webserver_port=0
--tserver_master_addrs=127.13.157.190:42857
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31524
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.157.131
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790304178325258-13942-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:46:57.464972 31524 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:46:57.465945 31524 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:46:57.475445 31530 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:46:57.475557 31532 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:46:57.475644 31524 server_base.cc:1034] running on GCE node
W20260925 02:46:57.475426 31529 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:46:57.476166 31524 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:46:57.476780 31524 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:46:57.477947 31524 hybrid_clock.cc:648] HybridClock initialized: now 1790304417477921 us; error 35 us; skew 500 ppm
Sep 25 02:46:57 dist-test-slave-kznw krb5kdc[31148](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304417, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:46:57.480583 31524 init.cc:377] Logged in from keytab as kudu/127.13.157.131@KRBTEST.COM (short username kudu)
I20260925 02:46:57.481389 31524 webserver.cc:460] Webserver started at http://127.13.157.131:43527/ using document root <none> and password file <none>
I20260925 02:46:57.481906 31524 fs_manager.cc:359] Metadata directory not provided
I20260925 02:46:57.481969 31524 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:46:57.482151 31524 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:46:57.484162 31524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790304178325258-13942-0/minicluster-data/ts-2/data/instance:
uuid: "bbec2c37e4b64af9885b274037bcc812"
format_stamp: "Formatted at 2026-09-25 02:46:57 on dist-test-slave-kznw"
server_key: "63c910d3899b4589a20dc02b8ecf91d9"
server_key_iv: "75b6d85c12263cc56297b3273d59e586"
server_key_version: "encryptionkey@0"
I20260925 02:46:57.484699 31524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790304178325258-13942-0/minicluster-data/ts-2/wal/instance:
uuid: "bbec2c37e4b64af9885b274037bcc812"
format_stamp: "Formatted at 2026-09-25 02:46:57 on dist-test-slave-kznw"
server_key: "63c910d3899b4589a20dc02b8ecf91d9"
server_key_iv: "75b6d85c12263cc56297b3273d59e586"
server_key_version: "encryptionkey@0"
I20260925 02:46:57.488154 31524 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260925 02:46:57.490406 31539 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:46:57.491593 31524 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260925 02:46:57.491698 31524 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790304178325258-13942-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790304178325258-13942-0/minicluster-data/ts-2/wal
uuid: "bbec2c37e4b64af9885b274037bcc812"
format_stamp: "Formatted at 2026-09-25 02:46:57 on dist-test-slave-kznw"
server_key: "63c910d3899b4589a20dc02b8ecf91d9"
server_key_iv: "75b6d85c12263cc56297b3273d59e586"
server_key_version: "encryptionkey@0"
I20260925 02:46:57.491784 31524 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790304178325258-13942-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790304178325258-13942-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790304178325258-13942-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:46:57.517228 31524 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:46:57.520123 31524 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:46:57.520293 31524 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:46:57.520854 31524 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:46:57.521664 31524 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:46:57.521709 31524 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:46:57.521754 31524 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:46:57.521782 31524 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:46:57.530237 31524 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.131:44563
I20260925 02:46:57.530267 31652 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.131:44563 every 8 connection(s)
I20260925 02:46:57.531116 31524 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790304178325258-13942-0/minicluster-data/ts-2/data/info.pb
I20260925 02:46:57.532889 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 31524
I20260925 02:46:57.533002 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790304178325258-13942-0/minicluster-data/ts-2/wal/instance
I20260925 02:46:57.533283 13942 external_mini_cluster.cc:1428] Setting key 49e33af9a3b16fa38827ea01a4e5bbf3
Sep 25 02:46:57 dist-test-slave-kznw krb5kdc[31148](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304417, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.131@KRBTEST.COM for kudu/127.13.157.190@KRBTEST.COM
I20260925 02:46:57.543815 31232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:57.532894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.131:35709 (local address 127.13.157.190:42857)
0925 02:46:57.533051 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0925 02:46:57.533056 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:46:57.533901 (+ 845us) server_negotiation.cc:407] Connection header received
0925 02:46:57.535200 (+ 1299us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:46:57.535204 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:46:57.535255 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:46:57.535328 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:46:57.536420 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:46:57.536945 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:46:57.538094 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:46:57.538297 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:46:57.538365 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:46:57.540813 (+ 2448us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:46:57.540816 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:46:57.540819 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:46:57.540845 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:46:57.541954 (+ 1109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:46:57.542391 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:46:57.542395 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:46:57.542396 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:46:57.542471 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:46:57.542767 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:46:57.542770 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:46:57.542772 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:46:57.542976 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:46:57.543083 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0925 02:46:57.543527 (+ 444us) server_negotiation.cc:299] Negotiation successful
0925 02:46:57.543653 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260925 02:46:57.544533 31655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:57.533274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:42857 (local address 127.13.157.131:35709)
0925 02:46:57.533748 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0925 02:46:57.533800 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0925 02:46:57.535032 (+ 1232us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:46:57.535477 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:46:57.535484 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:46:57.535828 (+ 344us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:46:57.536292 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:46:57.536305 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:46:57.537062 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:46:57.537065 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:46:57.537980 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:46:57.537990 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:46:57.538433 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:46:57.538436 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:46:57.538621 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:46:57.539217 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:46:57.539232 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:46:57.540665 (+ 1433us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:46:57.542054 (+ 1389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:46:57.542059 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:46:57.542073 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:46:57.542303 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:46:57.542554 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:46:57.542557 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:46:57.542560 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:46:57.542683 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:46:57.543080 (+ 397us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:46:57.543086 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:46:57.543312 (+ 226us) client_negotiation.cc:769] Sending connection context
0925 02:46:57.543470 (+ 158us) client_negotiation.cc:240] Negotiation successful
0925 02:46:57.543781 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":121,"threads_started":1}
I20260925 02:46:57.545755 31653 heartbeater.cc:347] Connected to a master server at 127.13.157.190:42857
I20260925 02:46:57.546095 31653 heartbeater.cc:464] Registering TS with master...
I20260925 02:46:57.546696 31653 heartbeater.cc:511] Master 127.13.157.190:42857 requested a full tablet report, sending...
I20260925 02:46:57.547803 31193 ts_manager.cc:194] Registered new tserver with Master: bbec2c37e4b64af9885b274037bcc812 (127.13.157.131:44563)
I20260925 02:46:57.548358 31193 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.157.131@KRBTEST.COM'} at 127.13.157.131:35709
I20260925 02:46:57.559345 13942 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 02:46:57.568228 31232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:57.561323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32788 (local address 127.13.157.190:42857)
0925 02:46:57.561456 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0925 02:46:57.561460 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:46:57.561622 (+ 162us) server_negotiation.cc:407] Connection header received
0925 02:46:57.561748 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:46:57.561750 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:46:57.561789 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:46:57.561848 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:46:57.562534 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:46:57.563099 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:46:57.563944 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:46:57.564140 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:46:57.564205 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:46:57.565625 (+ 1420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:46:57.565628 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:46:57.565630 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:46:57.565647 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:46:57.566783 (+ 1136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:46:57.567250 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:46:57.567253 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:46:57.567254 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:46:57.567326 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:46:57.567522 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:46:57.567524 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:46:57.567525 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:46:57.567748 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:46:57.567846 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0925 02:46:57.568009 (+ 163us) server_negotiation.cc:299] Negotiation successful
0925 02:46:57.568095 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260925 02:46:57.571061 31193 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32788:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 02:46:57.574030 31193 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 02:46:57.588027 31669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:57.583223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38724 (local address 127.13.157.130:46235)
0925 02:46:57.583583 (+ 360us) server_negotiation.cc:206] Beginning negotiation
0925 02:46:57.583593 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 02:46:57.583612 (+ 19us) server_negotiation.cc:407] Connection header received
0925 02:46:57.583683 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:46:57.583688 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:46:57.583851 (+ 163us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:46:57.584006 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:46:57.584439 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:46:57.585465 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:46:57.586819 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:46:57.587381 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:46:57.587490 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:46:57.587622 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0925 02:46:57.587780 (+ 158us) server_negotiation.cc:299] Negotiation successful
0925 02:46:57.587872 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":275,"thread_start_us":87,"threads_started":1}
I20260925 02:46:57.588020 31666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:57.582124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.129:42795 (local address 127.0.0.1:36718)
0925 02:46:57.582600 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0925 02:46:57.582645 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0925 02:46:57.582954 (+ 309us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:46:57.583697 (+ 743us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:46:57.583703 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:46:57.583726 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:46:57.583857 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:46:57.583864 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:46:57.585330 (+ 1466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:46:57.585333 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:46:57.586434 (+ 1101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:46:57.586443 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:46:57.587554 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:46:57.587558 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:46:57.587661 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:46:57.587682 (+ 21us) client_negotiation.cc:769] Sending connection context
0925 02:46:57.587742 (+ 60us) client_negotiation.cc:240] Negotiation successful
0925 02:46:57.587874 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"mutex_wait_us":55,"thread_start_us":71,"threads_started":1}
I20260925 02:46:57.588042 31665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:57.582262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36718 (local address 127.13.157.129:42795)
0925 02:46:57.583177 (+ 915us) server_negotiation.cc:206] Beginning negotiation
0925 02:46:57.583195 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0925 02:46:57.583219 (+ 24us) server_negotiation.cc:407] Connection header received
0925 02:46:57.583294 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:46:57.583299 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:46:57.583530 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:46:57.584008 (+ 478us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:46:57.584137 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:46:57.585205 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:46:57.586550 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:46:57.587416 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:46:57.587646 (+ 230us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:46:57.587769 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0925 02:46:57.587818 (+ 49us) server_negotiation.cc:299] Negotiation successful
0925 02:46:57.587901 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":815,"thread_start_us":92,"threads_started":1}
I20260925 02:46:57.588145 31667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:57.582820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.131:44563 (local address 127.0.0.1:60990)
0925 02:46:57.583257 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0925 02:46:57.583269 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 02:46:57.583349 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:46:57.583990 (+ 641us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:46:57.583993 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:46:57.584009 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:46:57.584123 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:46:57.584130 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:46:57.585142 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:46:57.585145 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:46:57.586523 (+ 1378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:46:57.586534 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:46:57.587562 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:46:57.587565 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:46:57.587859 (+ 294us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:46:57.587877 (+ 18us) client_negotiation.cc:769] Sending connection context
0925 02:46:57.587920 (+ 43us) client_negotiation.cc:240] Negotiation successful
0925 02:46:57.587978 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":379,"mutex_wait_us":43,"thread_start_us":111,"threads_started":1}
I20260925 02:46:57.588507 31664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:57.582245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.130:46235 (local address 127.0.0.1:38724)
0925 02:46:57.582799 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0925 02:46:57.582812 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 02:46:57.582891 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:46:57.584226 (+ 1335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:46:57.584229 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:46:57.584246 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:46:57.584328 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:46:57.584334 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:46:57.585576 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:46:57.585579 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:46:57.586717 (+ 1138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:46:57.586724 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:46:57.587490 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:46:57.587493 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:46:57.587600 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:46:57.587635 (+ 35us) client_negotiation.cc:769] Sending connection context
0925 02:46:57.587730 (+ 95us) client_negotiation.cc:240] Negotiation successful
0925 02:46:57.588384 (+ 654us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":498,"mutex_wait_us":4,"thread_start_us":100,"threads_started":1}
I20260925 02:46:57.588795 31668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:57.583003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60990 (local address 127.13.157.131:44563)
0925 02:46:57.583603 (+ 600us) server_negotiation.cc:206] Beginning negotiation
0925 02:46:57.583616 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0925 02:46:57.583637 (+ 21us) server_negotiation.cc:407] Connection header received
0925 02:46:57.583708 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:46:57.583714 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:46:57.583862 (+ 148us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:46:57.584005 (+ 143us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:46:57.584220 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:46:57.585001 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:46:57.586636 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:46:57.587416 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:46:57.588431 (+ 1015us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:46:57.588543 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0925 02:46:57.588597 (+ 54us) server_negotiation.cc:299] Negotiation successful
0925 02:46:57.588670 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":470,"thread_start_us":145,"threads_started":1}
I20260925 02:46:57.591060 31452 tablet_service.cc:1467] Processing CreateTablet for tablet 53e95a98b0ef4b159e23dc0cfcc1d96e (DEFAULT_TABLE table=test-table [id=60d6c85d3b654351a797dac318344c06]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:46:57.591193 31587 tablet_service.cc:1467] Processing CreateTablet for tablet 53e95a98b0ef4b159e23dc0cfcc1d96e (DEFAULT_TABLE table=test-table [id=60d6c85d3b654351a797dac318344c06]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:46:57.591194 31318 tablet_service.cc:1467] Processing CreateTablet for tablet 53e95a98b0ef4b159e23dc0cfcc1d96e (DEFAULT_TABLE table=test-table [id=60d6c85d3b654351a797dac318344c06]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:46:57.592028 31452 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 53e95a98b0ef4b159e23dc0cfcc1d96e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:46:57.592171 31587 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 53e95a98b0ef4b159e23dc0cfcc1d96e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:46:57.592171 31318 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 53e95a98b0ef4b159e23dc0cfcc1d96e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:46:57.598027 31670 tablet_bootstrap.cc:492] T 53e95a98b0ef4b159e23dc0cfcc1d96e P 45b1e950347a4fc8b9ca963fa3841394: Bootstrap starting.
I20260925 02:46:57.599490 31672 tablet_bootstrap.cc:492] T 53e95a98b0ef4b159e23dc0cfcc1d96e P d59e88b0215b4834ac7f76941b56bcc6: Bootstrap starting.
I20260925 02:46:57.600766 31670 tablet_bootstrap.cc:654] T 53e95a98b0ef4b159e23dc0cfcc1d96e P 45b1e950347a4fc8b9ca963fa3841394: Neither blocks nor log segments found. Creating new log.
I20260925 02:46:57.601248 31672 tablet_bootstrap.cc:654] T 53e95a98b0ef4b159e23dc0cfcc1d96e P d59e88b0215b4834ac7f76941b56bcc6: Neither blocks nor log segments found. Creating new log.
I20260925 02:46:57.601784 31670 log.cc:824] T 53e95a98b0ef4b159e23dc0cfcc1d96e P 45b1e950347a4fc8b9ca963fa3841394: Log is configured to *not* fsync() on all Append() calls
I20260925 02:46:57.601877 31672 log.cc:824] T 53e95a98b0ef4b159e23dc0cfcc1d96e P d59e88b0215b4834ac7f76941b56bcc6: Log is configured to *not* fsync() on all Append() calls
I20260925 02:46:57.602437 31671 tablet_bootstrap.cc:492] T 53e95a98b0ef4b159e23dc0cfcc1d96e P bbec2c37e4b64af9885b274037bcc812: Bootstrap starting.
I20260925 02:46:57.603572 31672 tablet_bootstrap.cc:492] T 53e95a98b0ef4b159e23dc0cfcc1d96e P d59e88b0215b4834ac7f76941b56bcc6: No bootstrap required, opened a new log
I20260925 02:46:57.603700 31670 tablet_bootstrap.cc:492] T 53e95a98b0ef4b159e23dc0cfcc1d96e P 45b1e950347a4fc8b9ca963fa3841394: No bootstrap required, opened a new log
I20260925 02:46:57.603746 31672 ts_tablet_manager.cc:1397] T 53e95a98b0ef4b159e23dc0cfcc1d96e P d59e88b0215b4834ac7f76941b56bcc6: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260925 02:46:57.603986 31670 ts_tablet_manager.cc:1397] T 53e95a98b0ef4b159e23dc0cfcc1d96e P 45b1e950347a4fc8b9ca963fa3841394: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260925 02:46:57.604480 31671 tablet_bootstrap.cc:654] T 53e95a98b0ef4b159e23dc0cfcc1d96e P bbec2c37e4b64af9885b274037bcc812: Neither blocks nor log segments found. Creating new log.
I20260925 02:46:57.605661 31671 log.cc:824] T 53e95a98b0ef4b159e23dc0cfcc1d96e P bbec2c37e4b64af9885b274037bcc812: Log is configured to *not* fsync() on all Append() calls
I20260925 02:46:57.607221 31672 raft_consensus.cc:348] T 53e95a98b0ef4b159e23dc0cfcc1d96e P d59e88b0215b4834ac7f76941b56bcc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d59e88b0215b4834ac7f76941b56bcc6" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 42795 } } peers { permanent_uuid: "45b1e950347a4fc8b9ca963fa3841394" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 46235 } } peers { permanent_uuid: "bbec2c37e4b64af9885b274037bcc812" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 44563 } }
I20260925 02:46:57.607426 31672 raft_consensus.cc:374] T 53e95a98b0ef4b159e23dc0cfcc1d96e P d59e88b0215b4834ac7f76941b56bcc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:46:57.607496 31672 raft_consensus.cc:729] T 53e95a98b0ef4b159e23dc0cfcc1d96e P d59e88b0215b4834ac7f76941b56bcc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d59e88b0215b4834ac7f76941b56bcc6, State: Initialized, Role: FOLLOWER
I20260925 02:46:57.607612 31671 tablet_bootstrap.cc:492] T 53e95a98b0ef4b159e23dc0cfcc1d96e P bbec2c37e4b64af9885b274037bcc812: No bootstrap required, opened a new log
I20260925 02:46:57.607621 31670 raft_consensus.cc:348] T 53e95a98b0ef4b159e23dc0cfcc1d96e P 45b1e950347a4fc8b9ca963fa3841394 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d59e88b0215b4834ac7f76941b56bcc6" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 42795 } } peers { permanent_uuid: "45b1e950347a4fc8b9ca963fa3841394" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 46235 } } peers { permanent_uuid: "bbec2c37e4b64af9885b274037bcc812" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 44563 } }
I20260925 02:46:57.607793 31671 ts_tablet_manager.cc:1397] T 53e95a98b0ef4b159e23dc0cfcc1d96e P bbec2c37e4b64af9885b274037bcc812: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260925 02:46:57.607853 31670 raft_consensus.cc:374] T 53e95a98b0ef4b159e23dc0cfcc1d96e P 45b1e950347a4fc8b9ca963fa3841394 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:46:57.607903 31670 raft_consensus.cc:729] T 53e95a98b0ef4b159e23dc0cfcc1d96e P 45b1e950347a4fc8b9ca963fa3841394 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45b1e950347a4fc8b9ca963fa3841394, State: Initialized, Role: FOLLOWER
I20260925 02:46:57.607923 31672 consensus_queue.cc:260] T 53e95a98b0ef4b159e23dc0cfcc1d96e P d59e88b0215b4834ac7f76941b56bcc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d59e88b0215b4834ac7f76941b56bcc6" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 42795 } } peers { permanent_uuid: "45b1e950347a4fc8b9ca963fa3841394" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 46235 } } peers { permanent_uuid: "bbec2c37e4b64af9885b274037bcc812" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 44563 } }
I20260925 02:46:57.608351 31670 consensus_queue.cc:260] T 53e95a98b0ef4b159e23dc0cfcc1d96e P 45b1e950347a4fc8b9ca963fa3841394 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d59e88b0215b4834ac7f76941b56bcc6" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 42795 } } peers { permanent_uuid: "45b1e950347a4fc8b9ca963fa3841394" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 46235 } } peers { permanent_uuid: "bbec2c37e4b64af9885b274037bcc812" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 44563 } }
I20260925 02:46:57.608546 31384 heartbeater.cc:503] Master 127.13.157.190:42857 was elected leader, sending a full tablet report...
I20260925 02:46:57.608901 31672 ts_tablet_manager.cc:1428] T 53e95a98b0ef4b159e23dc0cfcc1d96e P d59e88b0215b4834ac7f76941b56bcc6: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260925 02:46:57.609314 31670 ts_tablet_manager.cc:1428] T 53e95a98b0ef4b159e23dc0cfcc1d96e P 45b1e950347a4fc8b9ca963fa3841394: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260925 02:46:57.609522 31519 heartbeater.cc:503] Master 127.13.157.190:42857 was elected leader, sending a full tablet report...
I20260925 02:46:57.611483 31671 raft_consensus.cc:348] T 53e95a98b0ef4b159e23dc0cfcc1d96e P bbec2c37e4b64af9885b274037bcc812 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d59e88b0215b4834ac7f76941b56bcc6" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 42795 } } peers { permanent_uuid: "45b1e950347a4fc8b9ca963fa3841394" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 46235 } } peers { permanent_uuid: "bbec2c37e4b64af9885b274037bcc812" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 44563 } }
I20260925 02:46:57.611771 31671 raft_consensus.cc:374] T 53e95a98b0ef4b159e23dc0cfcc1d96e P bbec2c37e4b64af9885b274037bcc812 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:46:57.611840 31671 raft_consensus.cc:729] T 53e95a98b0ef4b159e23dc0cfcc1d96e P bbec2c37e4b64af9885b274037bcc812 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bbec2c37e4b64af9885b274037bcc812, State: Initialized, Role: FOLLOWER
I20260925 02:46:57.612219 31671 consensus_queue.cc:260] T 53e95a98b0ef4b159e23dc0cfcc1d96e P bbec2c37e4b64af9885b274037bcc812 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d59e88b0215b4834ac7f76941b56bcc6" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 42795 } } peers { permanent_uuid: "45b1e950347a4fc8b9ca963fa3841394" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 46235 } } peers { permanent_uuid: "bbec2c37e4b64af9885b274037bcc812" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 44563 } }
I20260925 02:46:57.613095 31671 ts_tablet_manager.cc:1428] T 53e95a98b0ef4b159e23dc0cfcc1d96e P bbec2c37e4b64af9885b274037bcc812: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260925 02:46:57.613314 31653 heartbeater.cc:503] Master 127.13.157.190:42857 was elected leader, sending a full tablet report...
I20260925 02:46:57.619068 31676 raft_consensus.cc:482] T 53e95a98b0ef4b159e23dc0cfcc1d96e P d59e88b0215b4834ac7f76941b56bcc6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 02:46:57.619234 31676 raft_consensus.cc:504] T 53e95a98b0ef4b159e23dc0cfcc1d96e P d59e88b0215b4834ac7f76941b56bcc6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d59e88b0215b4834ac7f76941b56bcc6" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 42795 } } peers { permanent_uuid: "45b1e950347a4fc8b9ca963fa3841394" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 46235 } } peers { permanent_uuid: "bbec2c37e4b64af9885b274037bcc812" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 44563 } }
I20260925 02:46:57.620201 31676 leader_election.cc:288] T 53e95a98b0ef4b159e23dc0cfcc1d96e P d59e88b0215b4834ac7f76941b56bcc6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45b1e950347a4fc8b9ca963fa3841394 (127.13.157.130:46235), bbec2c37e4b64af9885b274037bcc812 (127.13.157.131:44563)
I20260925 02:46:57.623895 31669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:57.620517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.129:47063 (local address 127.13.157.130:46235)
0925 02:46:57.620650 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0925 02:46:57.620654 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:46:57.620882 (+ 228us) server_negotiation.cc:407] Connection header received
0925 02:46:57.621083 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:46:57.621086 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:46:57.621135 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:46:57.621213 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:46:57.621469 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:46:57.621965 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:46:57.623001 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:46:57.623401 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:46:57.623506 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:46:57.623560 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0925 02:46:57.623719 (+ 159us) server_negotiation.cc:299] Negotiation successful
0925 02:46:57.623782 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260925 02:46:57.623924 31679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:57.620514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.130:46235 (local address 127.13.157.129:47063)
0925 02:46:57.620798 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0925 02:46:57.620814 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 02:46:57.620952 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:46:57.621238 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:46:57.621241 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:46:57.621260 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:46:57.621373 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:46:57.621378 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:46:57.622063 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:46:57.622067 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:46:57.622878 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:46:57.622888 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:46:57.623514 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:46:57.623517 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:46:57.623615 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:46:57.623633 (+ 18us) client_negotiation.cc:769] Sending connection context
0925 02:46:57.623697 (+ 64us) client_negotiation.cc:240] Negotiation successful
0925 02:46:57.623761 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213,"mutex_wait_us":3,"thread_start_us":117,"threads_started":1}
I20260925 02:46:57.624488 31668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:57.620998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.129:42299 (local address 127.13.157.131:44563)
0925 02:46:57.621102 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0925 02:46:57.621106 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:46:57.621273 (+ 167us) server_negotiation.cc:407] Connection header received
0925 02:46:57.621392 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:46:57.621408 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:46:57.621457 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:46:57.621524 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:46:57.621781 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:46:57.622284 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:46:57.623375 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:46:57.623785 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:46:57.623901 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:46:57.623952 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0925 02:46:57.624299 (+ 347us) server_negotiation.cc:299] Negotiation successful
0925 02:46:57.624371 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260925 02:46:57.624491 31680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:57.620932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.131:44563 (local address 127.13.157.129:42299)
0925 02:46:57.621187 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0925 02:46:57.621197 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 02:46:57.621293 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:46:57.621541 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:46:57.621545 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:46:57.621562 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:46:57.621679 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:46:57.621685 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:46:57.622393 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:46:57.622397 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:46:57.623244 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:46:57.623255 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:46:57.623907 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:46:57.623911 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:46:57.624176 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:46:57.624196 (+ 20us) client_negotiation.cc:769] Sending connection context
0925 02:46:57.624270 (+ 74us) client_negotiation.cc:240] Negotiation successful
0925 02:46:57.624340 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":95,"threads_started":1}
I20260925 02:46:57.624508 31472 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "53e95a98b0ef4b159e23dc0cfcc1d96e" candidate_uuid: "d59e88b0215b4834ac7f76941b56bcc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45b1e950347a4fc8b9ca963fa3841394" is_pre_election: true
I20260925 02:46:57.624780 31472 raft_consensus.cc:2454] T 53e95a98b0ef4b159e23dc0cfcc1d96e P 45b1e950347a4fc8b9ca963fa3841394 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d59e88b0215b4834ac7f76941b56bcc6 in term 0.
I20260925 02:46:57.625066 31607 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "53e95a98b0ef4b159e23dc0cfcc1d96e" candidate_uuid: "d59e88b0215b4834ac7f76941b56bcc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbec2c37e4b64af9885b274037bcc812" is_pre_election: true
I20260925 02:46:57.625238 31272 leader_election.cc:302] T 53e95a98b0ef4b159e23dc0cfcc1d96e P d59e88b0215b4834ac7f76941b56bcc6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45b1e950347a4fc8b9ca963fa3841394, d59e88b0215b4834ac7f76941b56bcc6; no voters:
I20260925 02:46:57.625344 31607 raft_consensus.cc:2454] T 53e95a98b0ef4b159e23dc0cfcc1d96e P bbec2c37e4b64af9885b274037bcc812 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d59e88b0215b4834ac7f76941b56bcc6 in term 0.
I20260925 02:46:57.625557 31676 raft_consensus.cc:2781] T 53e95a98b0ef4b159e23dc0cfcc1d96e P d59e88b0215b4834ac7f76941b56bcc6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 02:46:57.625650 31676 raft_consensus.cc:482] T 53e95a98b0ef4b159e23dc0cfcc1d96e P d59e88b0215b4834ac7f76941b56bcc6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 02:46:57.625706 31676 raft_consensus.cc:3037] T 53e95a98b0ef4b159e23dc0cfcc1d96e P d59e88b0215b4834ac7f76941b56bcc6 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:46:57.626994 31676 raft_consensus.cc:504] T 53e95a98b0ef4b159e23dc0cfcc1d96e P d59e88b0215b4834ac7f76941b56bcc6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d59e88b0215b4834ac7f76941b56bcc6" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 42795 } } peers { permanent_uuid: "45b1e950347a4fc8b9ca963fa3841394" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 46235 } } peers { permanent_uuid: "bbec2c37e4b64af9885b274037bcc812" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 44563 } }
I20260925 02:46:57.627336 31676 leader_election.cc:288] T 53e95a98b0ef4b159e23dc0cfcc1d96e P d59e88b0215b4834ac7f76941b56bcc6 [CANDIDATE]: Term 1 election: Requested vote from peers 45b1e950347a4fc8b9ca963fa3841394 (127.13.157.130:46235), bbec2c37e4b64af9885b274037bcc812 (127.13.157.131:44563)
I20260925 02:46:57.627646 31472 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "53e95a98b0ef4b159e23dc0cfcc1d96e" candidate_uuid: "d59e88b0215b4834ac7f76941b56bcc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45b1e950347a4fc8b9ca963fa3841394"
I20260925 02:46:57.627687 31607 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "53e95a98b0ef4b159e23dc0cfcc1d96e" candidate_uuid: "d59e88b0215b4834ac7f76941b56bcc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbec2c37e4b64af9885b274037bcc812"
I20260925 02:46:57.627748 31472 raft_consensus.cc:3037] T 53e95a98b0ef4b159e23dc0cfcc1d96e P 45b1e950347a4fc8b9ca963fa3841394 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:46:57.627810 31607 raft_consensus.cc:3037] T 53e95a98b0ef4b159e23dc0cfcc1d96e P bbec2c37e4b64af9885b274037bcc812 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:46:57.628789 31472 raft_consensus.cc:2454] T 53e95a98b0ef4b159e23dc0cfcc1d96e P 45b1e950347a4fc8b9ca963fa3841394 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d59e88b0215b4834ac7f76941b56bcc6 in term 1.
I20260925 02:46:57.628847 31607 raft_consensus.cc:2454] T 53e95a98b0ef4b159e23dc0cfcc1d96e P bbec2c37e4b64af9885b274037bcc812 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d59e88b0215b4834ac7f76941b56bcc6 in term 1.
I20260925 02:46:57.629060 31272 leader_election.cc:302] T 53e95a98b0ef4b159e23dc0cfcc1d96e P d59e88b0215b4834ac7f76941b56bcc6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45b1e950347a4fc8b9ca963fa3841394, d59e88b0215b4834ac7f76941b56bcc6; no voters:
I20260925 02:46:57.629227 31676 raft_consensus.cc:2781] T 53e95a98b0ef4b159e23dc0cfcc1d96e P d59e88b0215b4834ac7f76941b56bcc6 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:46:57.629422 31676 raft_consensus.cc:686] T 53e95a98b0ef4b159e23dc0cfcc1d96e P d59e88b0215b4834ac7f76941b56bcc6 [term 1 LEADER]: Becoming Leader. State: Replica: d59e88b0215b4834ac7f76941b56bcc6, State: Running, Role: LEADER
I20260925 02:46:57.629756 31676 consensus_queue.cc:237] T 53e95a98b0ef4b159e23dc0cfcc1d96e P d59e88b0215b4834ac7f76941b56bcc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d59e88b0215b4834ac7f76941b56bcc6" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 42795 } } peers { permanent_uuid: "45b1e950347a4fc8b9ca963fa3841394" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 46235 } } peers { permanent_uuid: "bbec2c37e4b64af9885b274037bcc812" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 44563 } }
I20260925 02:46:57.632682 31191 catalog_manager.cc:5495] T 53e95a98b0ef4b159e23dc0cfcc1d96e P d59e88b0215b4834ac7f76941b56bcc6 reported cstate change: term changed from 0 to 1, leader changed from <none> to d59e88b0215b4834ac7f76941b56bcc6 (127.13.157.129). New cstate: current_term: 1 leader_uuid: "d59e88b0215b4834ac7f76941b56bcc6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d59e88b0215b4834ac7f76941b56bcc6" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 42795 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45b1e950347a4fc8b9ca963fa3841394" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 46235 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bbec2c37e4b64af9885b274037bcc812" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 44563 } health_report { overall_health: UNKNOWN } } }
I20260925 02:46:57.644671 31665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:57.641338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36730 (local address 127.13.157.129:42795)
0925 02:46:57.641451 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0925 02:46:57.641455 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:46:57.641468 (+ 13us) server_negotiation.cc:407] Connection header received
0925 02:46:57.641615 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:46:57.641617 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:46:57.641668 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:46:57.641751 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 02:46:57.641977 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:46:57.642458 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:46:57.643372 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:46:57.643580 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:46:57.643663 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:46:57.643927 (+ 264us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 02:46:57.644309 (+ 382us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 02:46:57.644412 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0925 02:46:57.644479 (+ 67us) server_negotiation.cc:299] Negotiation successful
0925 02:46:57.644532 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260925 02:46:57.650202 31472 raft_consensus.cc:1261] T 53e95a98b0ef4b159e23dc0cfcc1d96e P 45b1e950347a4fc8b9ca963fa3841394 [term 1 FOLLOWER]: Refusing update from remote peer d59e88b0215b4834ac7f76941b56bcc6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:46:57.650202 31607 raft_consensus.cc:1261] T 53e95a98b0ef4b159e23dc0cfcc1d96e P bbec2c37e4b64af9885b274037bcc812 [term 1 FOLLOWER]: Refusing update from remote peer d59e88b0215b4834ac7f76941b56bcc6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:46:57.650866 31681 consensus_queue.cc:1035] T 53e95a98b0ef4b159e23dc0cfcc1d96e P d59e88b0215b4834ac7f76941b56bcc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45b1e950347a4fc8b9ca963fa3841394" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 46235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:46:57.651118 31676 consensus_queue.cc:1035] T 53e95a98b0ef4b159e23dc0cfcc1d96e P d59e88b0215b4834ac7f76941b56bcc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bbec2c37e4b64af9885b274037bcc812" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 44563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:46:57.660406 31686 mvcc.cc:205] Tried to move back new op lower bound from 7333086894688227328 to 7333086894613938176. Current Snapshot: MvccSnapshot[applied={T|T < 7333086894688227328}]
I20260925 02:46:57.667798 31193 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32788:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 02:46:57.667944 31193 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32788:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 02:46:57.670122 31193 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 63515883fb3141769fde9dbae17dd934: Sending DeleteTablet for 3 replicas of tablet 53e95a98b0ef4b159e23dc0cfcc1d96e
I20260925 02:46:57.670783 31318 tablet_service.cc:1514] Processing DeleteTablet for tablet 53e95a98b0ef4b159e23dc0cfcc1d96e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:46:57 UTC) from {username='kudu', principal='kudu/127.13.157.190@KRBTEST.COM'} at 127.0.0.1:36718
I20260925 02:46:57.670984 31587 tablet_service.cc:1514] Processing DeleteTablet for tablet 53e95a98b0ef4b159e23dc0cfcc1d96e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:46:57 UTC) from {username='kudu', principal='kudu/127.13.157.190@KRBTEST.COM'} at 127.0.0.1:60990
I20260925 02:46:57.671087 31452 tablet_service.cc:1514] Processing DeleteTablet for tablet 53e95a98b0ef4b159e23dc0cfcc1d96e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:46:57 UTC) from {username='kudu', principal='kudu/127.13.157.190@KRBTEST.COM'} at 127.0.0.1:38724
I20260925 02:46:57.671451 31694 tablet_replica.cc:330] T 53e95a98b0ef4b159e23dc0cfcc1d96e P d59e88b0215b4834ac7f76941b56bcc6: stopping tablet replica
I20260925 02:46:57.671790 31694 raft_consensus.cc:2229] T 53e95a98b0ef4b159e23dc0cfcc1d96e P d59e88b0215b4834ac7f76941b56bcc6 [term 1 LEADER]: Raft consensus shutting down.
I20260925 02:46:57.672171 31694 raft_consensus.cc:2258] T 53e95a98b0ef4b159e23dc0cfcc1d96e P d59e88b0215b4834ac7f76941b56bcc6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:46:57.672214 31696 tablet_replica.cc:330] T 53e95a98b0ef4b159e23dc0cfcc1d96e P 45b1e950347a4fc8b9ca963fa3841394: stopping tablet replica
I20260925 02:46:57.672421 31695 tablet_replica.cc:330] T 53e95a98b0ef4b159e23dc0cfcc1d96e P bbec2c37e4b64af9885b274037bcc812: stopping tablet replica
I20260925 02:46:57.672585 31696 raft_consensus.cc:2229] T 53e95a98b0ef4b159e23dc0cfcc1d96e P 45b1e950347a4fc8b9ca963fa3841394 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 02:46:57.672611 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 31255
I20260925 02:46:57.673378 31695 raft_consensus.cc:2229] T 53e95a98b0ef4b159e23dc0cfcc1d96e P bbec2c37e4b64af9885b274037bcc812 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 02:46:57.673707 31695 raft_consensus.cc:2258] T 53e95a98b0ef4b159e23dc0cfcc1d96e P bbec2c37e4b64af9885b274037bcc812 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:46:57.674216 31696 raft_consensus.cc:2258] T 53e95a98b0ef4b159e23dc0cfcc1d96e P 45b1e950347a4fc8b9ca963fa3841394 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:46:57.675731 31696 ts_tablet_manager.cc:1905] T 53e95a98b0ef4b159e23dc0cfcc1d96e P 45b1e950347a4fc8b9ca963fa3841394: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:46:57.675731 31695 ts_tablet_manager.cc:1905] T 53e95a98b0ef4b159e23dc0cfcc1d96e P bbec2c37e4b64af9885b274037bcc812: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:46:57.678388 31695 ts_tablet_manager.cc:1918] T 53e95a98b0ef4b159e23dc0cfcc1d96e P bbec2c37e4b64af9885b274037bcc812: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 02:46:57.678388 31696 ts_tablet_manager.cc:1918] T 53e95a98b0ef4b159e23dc0cfcc1d96e P 45b1e950347a4fc8b9ca963fa3841394: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 02:46:57.678493 31695 log.cc:1196] T 53e95a98b0ef4b159e23dc0cfcc1d96e P bbec2c37e4b64af9885b274037bcc812: Deleting WAL directory at /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790304178325258-13942-0/minicluster-data/ts-2/wal/wals/53e95a98b0ef4b159e23dc0cfcc1d96e
I20260925 02:46:57.678493 31696 log.cc:1196] T 53e95a98b0ef4b159e23dc0cfcc1d96e P 45b1e950347a4fc8b9ca963fa3841394: Deleting WAL directory at /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790304178325258-13942-0/minicluster-data/ts-1/wal/wals/53e95a98b0ef4b159e23dc0cfcc1d96e
I20260925 02:46:57.678824 31696 ts_tablet_manager.cc:1939] T 53e95a98b0ef4b159e23dc0cfcc1d96e P 45b1e950347a4fc8b9ca963fa3841394: Deleting consensus metadata
I20260925 02:46:57.678824 31695 ts_tablet_manager.cc:1939] T 53e95a98b0ef4b159e23dc0cfcc1d96e P bbec2c37e4b64af9885b274037bcc812: Deleting consensus metadata
W20260925 02:46:57.679095 31179 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.157.129:42795 (error 108)
I20260925 02:46:57.680097 31664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:57.679818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.129:42795 (local address 127.0.0.1:36734)
0925 02:46:57.679946 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0925 02:46:57.680013 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.129:42795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260925 02:46:57.680101 31178 catalog_manager.cc:4841] TS 45b1e950347a4fc8b9ca963fa3841394 (127.13.157.130:46235): tablet 53e95a98b0ef4b159e23dc0cfcc1d96e (table test-table [id=60d6c85d3b654351a797dac318344c06]) successfully deleted
I20260925 02:46:57.680215 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 31389
W20260925 02:46:57.680269 31179 catalog_manager.cc:4584] TS d59e88b0215b4834ac7f76941b56bcc6 (127.13.157.129:42795): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 53e95a98b0ef4b159e23dc0cfcc1d96e: Network error: Client connection negotiation failed: client connection to 127.13.157.129:42795: connect: Connection refused (error 111)
I20260925 02:46:57.680321 31178 catalog_manager.cc:4841] TS bbec2c37e4b64af9885b274037bcc812 (127.13.157.131:44563): tablet 53e95a98b0ef4b159e23dc0cfcc1d96e (table test-table [id=60d6c85d3b654351a797dac318344c06]) successfully deleted
I20260925 02:46:57.685665 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 31524
I20260925 02:46:57.690173 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 31161
2026-09-25T02:46:57Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2156 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-09-25T02:46:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T02:46:57Z Disabled control of system clock
I20260925 02:46:57.717473 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790304178325258-13942-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790304178325258-13942-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790304178325258-13942-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.157.190:39467
--webserver_interface=127.13.157.190
--webserver_port=0
--builtin_ntp_servers=127.13.157.148:36235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.157.190:39467
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260925 02:46:57.823189 31701 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:46:57.823439 31701 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:46:57.823473 31701 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:46:57.827499 31701 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:46:57.827565 31701 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:46:57.827580 31701 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:46:57.827595 31701 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:46:57.827610 31701 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:46:57.832413 31701 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:36235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790304178325258-13942-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790304178325258-13942-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.157.190:39467
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.157.190:39467
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.157.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31701
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790304178325258-13942-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:46:57.833496 31701 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:46:57.834925 31701 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:46:57.843303 31706 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:46:57.843492 31709 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:46:57.843310 31707 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:46:57.843411 31701 server_base.cc:1034] running on GCE node
I20260925 02:46:57.844174 31701 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:46:57.845316 31701 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:46:57.846481 31701 hybrid_clock.cc:648] HybridClock initialized: now 1790304417846466 us; error 28 us; skew 500 ppm
I20260925 02:46:57.848485 31701 webserver.cc:460] Webserver started at http://127.13.157.190:43227/ using document root <none> and password file <none>
I20260925 02:46:57.849059 31701 fs_manager.cc:359] Metadata directory not provided
I20260925 02:46:57.849123 31701 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:46:57.849327 31701 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:46:57.851425 31701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790304178325258-13942-0/minicluster-data/master-0/data/instance:
uuid: "ea93940d6a304708817719383144178d"
format_stamp: "Formatted at 2026-09-25 02:46:57 on dist-test-slave-kznw"
server_key: "f3c03780f0a8625421ab274323d6fcde"
server_key_iv: "e487e73e3e5bfafe9984c92f10d3f2f5"
server_key_version: "encryptionkey@0"
I20260925 02:46:57.851975 31701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790304178325258-13942-0/minicluster-data/master-0/wal/instance:
uuid: "ea93940d6a304708817719383144178d"
format_stamp: "Formatted at 2026-09-25 02:46:57 on dist-test-slave-kznw"
server_key: "f3c03780f0a8625421ab274323d6fcde"
server_key_iv: "e487e73e3e5bfafe9984c92f10d3f2f5"
server_key_version: "encryptionkey@0"
I20260925 02:46:57.855564 31701 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260925 02:46:57.857868 31715 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:46:57.859170 31701 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260925 02:46:57.859282 31701 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790304178325258-13942-0/minicluster-data/master-0/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790304178325258-13942-0/minicluster-data/master-0/wal
uuid: "ea93940d6a304708817719383144178d"
format_stamp: "Formatted at 2026-09-25 02:46:57 on dist-test-slave-kznw"
server_key: "f3c03780f0a8625421ab274323d6fcde"
server_key_iv: "e487e73e3e5bfafe9984c92f10d3f2f5"
server_key_version: "encryptionkey@0"
I20260925 02:46:57.859390 31701 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790304178325258-13942-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790304178325258-13942-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790304178325258-13942-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:46:57.876013 31701 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:46:57.876716 31701 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:46:57.876893 31701 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:46:57.883518 31701 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.190:39467
I20260925 02:46:57.883524 31767 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.190:39467 every 8 connection(s)
I20260925 02:46:57.884454 31701 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
I20260925 02:46:57.887459 31768 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:46:57.892722 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 31701
I20260925 02:46:57.892858 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790304178325258-13942-0/minicluster-data/master-0/wal/instance
I20260925 02:46:57.893194 13942 external_mini_cluster.cc:1428] Setting key d9ea1daada82487e0b810d6909fcd6f4
I20260925 02:46:57.893177 31768 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea93940d6a304708817719383144178d: Bootstrap starting.
I20260925 02:46:57.895762 31768 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ea93940d6a304708817719383144178d: Neither blocks nor log segments found. Creating new log.
I20260925 02:46:57.896514 31768 log.cc:824] T 00000000000000000000000000000000 P ea93940d6a304708817719383144178d: Log is configured to *not* fsync() on all Append() calls
I20260925 02:46:57.898576 31768 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea93940d6a304708817719383144178d: No bootstrap required, opened a new log
I20260925 02:46:57.902647 31768 raft_consensus.cc:348] T 00000000000000000000000000000000 P ea93940d6a304708817719383144178d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea93940d6a304708817719383144178d" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 39467 } }
I20260925 02:46:57.902859 31771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:57.894631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47814 (local address 127.13.157.190:39467)
0925 02:46:57.895300 (+ 669us) server_negotiation.cc:206] Beginning negotiation
0925 02:46:57.895314 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0925 02:46:57.895350 (+ 36us) server_negotiation.cc:407] Connection header received
0925 02:46:57.896065 (+ 715us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:46:57.896085 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:46:57.896521 (+ 436us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:46:57.896829 (+ 308us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:46:57.897284 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:46:57.898337 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:46:57.899575 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:46:57.900058 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:46:57.900181 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:46:57.901020 (+ 839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:46:57.901025 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:46:57.901037 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:46:57.901419 (+ 382us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:46:57.901534 (+ 115us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:46:57.901555 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:46:57.901630 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0925 02:46:57.901788 (+ 158us) server_negotiation.cc:299] Negotiation successful
0925 02:46:57.902057 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":126,"threads_started":1}
I20260925 02:46:57.902870 31768 raft_consensus.cc:374] T 00000000000000000000000000000000 P ea93940d6a304708817719383144178d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:46:57.902976 31768 raft_consensus.cc:729] T 00000000000000000000000000000000 P ea93940d6a304708817719383144178d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea93940d6a304708817719383144178d, State: Initialized, Role: FOLLOWER
I20260925 02:46:57.903468 31768 consensus_queue.cc:260] T 00000000000000000000000000000000 P ea93940d6a304708817719383144178d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea93940d6a304708817719383144178d" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 39467 } }
I20260925 02:46:57.903645 31768 raft_consensus.cc:388] T 00000000000000000000000000000000 P ea93940d6a304708817719383144178d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 02:46:57.903709 31768 raft_consensus.cc:482] T 00000000000000000000000000000000 P ea93940d6a304708817719383144178d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 02:46:57.903800 31768 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ea93940d6a304708817719383144178d [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:46:57.904707 31768 raft_consensus.cc:504] T 00000000000000000000000000000000 P ea93940d6a304708817719383144178d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea93940d6a304708817719383144178d" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 39467 } }
I20260925 02:46:57.905023 31768 leader_election.cc:302] T 00000000000000000000000000000000 P ea93940d6a304708817719383144178d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ea93940d6a304708817719383144178d; no voters:
I20260925 02:46:57.905306 31768 leader_election.cc:288] T 00000000000000000000000000000000 P ea93940d6a304708817719383144178d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 02:46:57.905403 31773 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ea93940d6a304708817719383144178d [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:46:57.905602 31773 raft_consensus.cc:686] T 00000000000000000000000000000000 P ea93940d6a304708817719383144178d [term 1 LEADER]: Becoming Leader. State: Replica: ea93940d6a304708817719383144178d, State: Running, Role: LEADER
I20260925 02:46:57.905946 31773 consensus_queue.cc:237] T 00000000000000000000000000000000 P ea93940d6a304708817719383144178d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea93940d6a304708817719383144178d" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 39467 } }
I20260925 02:46:57.906441 31768 sys_catalog.cc:564] T 00000000000000000000000000000000 P ea93940d6a304708817719383144178d [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:46:57.907658 31775 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea93940d6a304708817719383144178d [sys.catalog]: SysCatalogTable state changed. Reason: New leader ea93940d6a304708817719383144178d. Latest consensus state: current_term: 1 leader_uuid: "ea93940d6a304708817719383144178d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea93940d6a304708817719383144178d" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 39467 } } }
I20260925 02:46:57.907691 31774 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea93940d6a304708817719383144178d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ea93940d6a304708817719383144178d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea93940d6a304708817719383144178d" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 39467 } } }
I20260925 02:46:57.907794 31775 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea93940d6a304708817719383144178d [sys.catalog]: This master's current role is: LEADER
I20260925 02:46:57.907859 31774 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea93940d6a304708817719383144178d [sys.catalog]: This master's current role is: LEADER
I20260925 02:46:57.908207 31782 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:46:57.911455 31782 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:46:57.916590 31782 catalog_manager.cc:1347] Generated new cluster ID: d5dcec61b50647aca373df556fbc06d2
I20260925 02:46:57.916656 31782 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 02:46:57.936254 31782 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 02:46:57.937183 31782 catalog_manager.cc:1504] Loading token signing keys...
I20260925 02:46:57.944408 31782 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ea93940d6a304708817719383144178d: Generated new TSK 0
I20260925 02:46:57.945096 31782 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 02:46:57.956826 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790304178325258-13942-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790304178325258-13942-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790304178325258-13942-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790304178325258-13942-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.157.129:0
--local_ip_for_outbound_sockets=127.13.157.129
--webserver_interface=127.13.157.129
--webserver_port=0
--tserver_master_addrs=127.13.157.190:39467
--builtin_ntp_servers=127.13.157.148:36235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260925 02:46:58.072383 31793 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:46:58.072638 31793 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:46:58.072675 31793 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:46:58.076615 31793 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:46:58.076665 31793 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:46:58.076725 31793 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.157.129
I20260925 02:46:58.081526 31793 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:36235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790304178325258-13942-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790304178325258-13942-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.157.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790304178325258-13942-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.157.129
--webserver_port=0
--tserver_master_addrs=127.13.157.190:39467
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31793
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.157.129
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790304178325258-13942-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:46:58.082654 31793 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:46:58.084100 31793 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:46:58.093689 31801 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:46:58.093693 31798 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:46:58.093771 31793 server_base.cc:1034] running on GCE node
W20260925 02:46:58.093892 31799 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:46:58.094455 31793 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:46:58.095142 31793 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:46:58.096315 31793 hybrid_clock.cc:648] HybridClock initialized: now 1790304418096291 us; error 34 us; skew 500 ppm
I20260925 02:46:58.098160 31793 webserver.cc:460] Webserver started at http://127.13.157.129:43393/ using document root <none> and password file <none>
I20260925 02:46:58.098719 31793 fs_manager.cc:359] Metadata directory not provided
I20260925 02:46:58.098788 31793 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:46:58.099042 31793 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:46:58.101084 31793 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790304178325258-13942-0/minicluster-data/ts-0/data/instance:
uuid: "179db8bcab274aa58074cb94838a3136"
format_stamp: "Formatted at 2026-09-25 02:46:58 on dist-test-slave-kznw"
server_key: "6c547d6528defd06c12e52f618b2710e"
server_key_iv: "4889d4f2006ac47f314e38ea5e82601d"
server_key_version: "encryptionkey@0"
I20260925 02:46:58.101617 31793 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790304178325258-13942-0/minicluster-data/ts-0/wal/instance:
uuid: "179db8bcab274aa58074cb94838a3136"
format_stamp: "Formatted at 2026-09-25 02:46:58 on dist-test-slave-kznw"
server_key: "6c547d6528defd06c12e52f618b2710e"
server_key_iv: "4889d4f2006ac47f314e38ea5e82601d"
server_key_version: "encryptionkey@0"
I20260925 02:46:58.105356 31793 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260925 02:46:58.107885 31807 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:46:58.109135 31793 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260925 02:46:58.109253 31793 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790304178325258-13942-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790304178325258-13942-0/minicluster-data/ts-0/wal
uuid: "179db8bcab274aa58074cb94838a3136"
format_stamp: "Formatted at 2026-09-25 02:46:58 on dist-test-slave-kznw"
server_key: "6c547d6528defd06c12e52f618b2710e"
server_key_iv: "4889d4f2006ac47f314e38ea5e82601d"
server_key_version: "encryptionkey@0"
I20260925 02:46:58.109357 31793 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790304178325258-13942-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790304178325258-13942-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790304178325258-13942-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:46:58.129652 31793 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:46:58.130334 31793 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:46:58.130515 31793 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:46:58.131117 31793 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:46:58.132097 31793 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:46:58.132146 31793 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:46:58.132201 31793 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:46:58.132227 31793 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:46:58.140712 31793 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.129:40115
I20260925 02:46:58.140754 31922 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.129:40115 every 8 connection(s)
I20260925 02:46:58.141602 31793 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790304178325258-13942-0/minicluster-data/ts-0/data/info.pb
I20260925 02:46:58.144199 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 31793
I20260925 02:46:58.144316 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790304178325258-13942-0/minicluster-data/ts-0/wal/instance
I20260925 02:46:58.144639 13942 external_mini_cluster.cc:1428] Setting key 467e574f02f4d72ceb0478dc32985b24
I20260925 02:46:58.146641 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790304178325258-13942-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790304178325258-13942-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790304178325258-13942-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790304178325258-13942-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.157.130:0
--local_ip_for_outbound_sockets=127.13.157.130
--webserver_interface=127.13.157.130
--webserver_port=0
--tserver_master_addrs=127.13.157.190:39467
--builtin_ntp_servers=127.13.157.148:36235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260925 02:46:58.151532 31771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:58.143359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.129:53111 (local address 127.13.157.190:39467)
0925 02:46:58.143529 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0925 02:46:58.143535 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 02:46:58.144749 (+ 1214us) server_negotiation.cc:407] Connection header received
0925 02:46:58.146201 (+ 1452us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:46:58.146203 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:46:58.146246 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:46:58.146321 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:46:58.147499 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:46:58.148019 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:46:58.149374 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:46:58.149635 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:46:58.149718 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:46:58.151039 (+ 1321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:46:58.151042 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:46:58.151044 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:46:58.151056 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:46:58.151074 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:46:58.151082 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:46:58.151139 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0925 02:46:58.151319 (+ 180us) server_negotiation.cc:299] Negotiation successful
0925 02:46:58.151376 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260925 02:46:58.152648 31925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:58.143765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:39467 (local address 127.13.157.129:53111)
0925 02:46:58.144549 (+ 784us) negotiation.cc:107] Waiting for socket to connect
0925 02:46:58.144605 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0925 02:46:58.145983 (+ 1378us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:46:58.146521 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:46:58.146531 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:46:58.146639 (+ 108us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:46:58.147345 (+ 706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:46:58.147363 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:46:58.148145 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:46:58.148148 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:46:58.149239 (+ 1091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:46:58.149252 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:46:58.149791 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:46:58.149795 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:46:58.150018 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:46:58.150705 (+ 687us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 02:46:58.150707 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:46:58.150765 (+ 58us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 02:46:58.150852 (+ 87us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:46:58.151164 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:46:58.151174 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:46:58.151181 (+ 7us) client_negotiation.cc:769] Sending connection context
0925 02:46:58.151317 (+ 136us) client_negotiation.cc:240] Negotiation successful
0925 02:46:58.151627 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":123,"threads_started":1}
I20260925 02:46:58.153901 31923 heartbeater.cc:347] Connected to a master server at 127.13.157.190:39467
I20260925 02:46:58.154117 31923 heartbeater.cc:464] Registering TS with master...
I20260925 02:46:58.154608 31923 heartbeater.cc:511] Master 127.13.157.190:39467 requested a full tablet report, sending...
I20260925 02:46:58.156090 31732 ts_manager.cc:194] Registered new tserver with Master: 179db8bcab274aa58074cb94838a3136 (127.13.157.129:40115)
I20260925 02:46:58.157166 31732 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.157.129:53111
W20260925 02:46:58.260171 31926 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:46:58.260443 31926 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:46:58.260499 31926 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:46:58.264813 31926 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:46:58.264878 31926 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:46:58.264956 31926 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.157.130
I20260925 02:46:58.271639 31926 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:36235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790304178325258-13942-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790304178325258-13942-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.157.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790304178325258-13942-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.157.130
--webserver_port=0
--tserver_master_addrs=127.13.157.190:39467
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31926
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.157.130
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790304178325258-13942-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:46:58.272797 31926 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:46:58.274161 31926 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:46:58.283797 31931 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:46:58.283797 31932 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:46:58.283898 31934 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:46:58.283959 31926 server_base.cc:1034] running on GCE node
I20260925 02:46:58.284473 31926 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:46:58.285110 31926 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:46:58.286278 31926 hybrid_clock.cc:648] HybridClock initialized: now 1790304418286257 us; error 29 us; skew 500 ppm
I20260925 02:46:58.288074 31926 webserver.cc:460] Webserver started at http://127.13.157.130:38573/ using document root <none> and password file <none>
I20260925 02:46:58.288614 31926 fs_manager.cc:359] Metadata directory not provided
I20260925 02:46:58.288679 31926 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:46:58.288885 31926 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:46:58.290961 31926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790304178325258-13942-0/minicluster-data/ts-1/data/instance:
uuid: "d73cab0ba40e4c6c842faa1ebd990715"
format_stamp: "Formatted at 2026-09-25 02:46:58 on dist-test-slave-kznw"
server_key: "66d20a91b71457582171eadd665d51af"
server_key_iv: "022a4bdcae1245b853633a44281bfaeb"
server_key_version: "encryptionkey@0"
I20260925 02:46:58.291512 31926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790304178325258-13942-0/minicluster-data/ts-1/wal/instance:
uuid: "d73cab0ba40e4c6c842faa1ebd990715"
format_stamp: "Formatted at 2026-09-25 02:46:58 on dist-test-slave-kznw"
server_key: "66d20a91b71457582171eadd665d51af"
server_key_iv: "022a4bdcae1245b853633a44281bfaeb"
server_key_version: "encryptionkey@0"
I20260925 02:46:58.295157 31926 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260925 02:46:58.297478 31940 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:46:58.298674 31926 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260925 02:46:58.298780 31926 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790304178325258-13942-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790304178325258-13942-0/minicluster-data/ts-1/wal
uuid: "d73cab0ba40e4c6c842faa1ebd990715"
format_stamp: "Formatted at 2026-09-25 02:46:58 on dist-test-slave-kznw"
server_key: "66d20a91b71457582171eadd665d51af"
server_key_iv: "022a4bdcae1245b853633a44281bfaeb"
server_key_version: "encryptionkey@0"
I20260925 02:46:58.298870 31926 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790304178325258-13942-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790304178325258-13942-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790304178325258-13942-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:46:58.325989 31926 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:46:58.326725 31926 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:46:58.326925 31926 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:46:58.327543 31926 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:46:58.328562 31926 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:46:58.328612 31926 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:46:58.328666 31926 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:46:58.328693 31926 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:46:58.337189 31926 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.130:36195
I20260925 02:46:58.337229 32053 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.130:36195 every 8 connection(s)
I20260925 02:46:58.338131 31926 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790304178325258-13942-0/minicluster-data/ts-1/data/info.pb
I20260925 02:46:58.343452 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 31926
I20260925 02:46:58.343593 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790304178325258-13942-0/minicluster-data/ts-1/wal/instance
I20260925 02:46:58.343883 13942 external_mini_cluster.cc:1428] Setting key 4cf820bb9d3e7d720b5bc0f74c777b85
I20260925 02:46:58.345824 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790304178325258-13942-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790304178325258-13942-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790304178325258-13942-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790304178325258-13942-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.157.131:0
--local_ip_for_outbound_sockets=127.13.157.131
--webserver_interface=127.13.157.131
--webserver_port=0
--tserver_master_addrs=127.13.157.190:39467
--builtin_ntp_servers=127.13.157.148:36235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260925 02:46:58.347543 31771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:58.340050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.130:33273 (local address 127.13.157.190:39467)
0925 02:46:58.340234 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0925 02:46:58.340237 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 02:46:58.341290 (+ 1053us) server_negotiation.cc:407] Connection header received
0925 02:46:58.342647 (+ 1357us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:46:58.342650 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:46:58.342699 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:46:58.342776 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:46:58.343793 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:46:58.344351 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:46:58.345652 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:46:58.345866 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:46:58.345953 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:46:58.347070 (+ 1117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:46:58.347075 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:46:58.347077 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:46:58.347092 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:46:58.347119 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:46:58.347130 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:46:58.347212 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0925 02:46:58.347376 (+ 164us) server_negotiation.cc:299] Negotiation successful
0925 02:46:58.347429 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260925 02:46:58.348706 32056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:58.340461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:39467 (local address 127.13.157.130:33273)
0925 02:46:58.341141 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0925 02:46:58.341185 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0925 02:46:58.342442 (+ 1257us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:46:58.342961 (+ 519us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:46:58.342970 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:46:58.343065 (+ 95us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:46:58.343638 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:46:58.343657 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:46:58.344465 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:46:58.344469 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:46:58.345534 (+ 1065us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:46:58.345544 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:46:58.346019 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:46:58.346023 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:46:58.346193 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:46:58.346760 (+ 567us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 02:46:58.346761 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:46:58.346802 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 02:46:58.346875 (+ 73us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:46:58.347225 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:46:58.347235 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:46:58.347240 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 02:46:58.347369 (+ 129us) client_negotiation.cc:240] Negotiation successful
0925 02:46:58.347677 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":113,"threads_started":1}
I20260925 02:46:58.349848 32054 heartbeater.cc:347] Connected to a master server at 127.13.157.190:39467
I20260925 02:46:58.350098 32054 heartbeater.cc:464] Registering TS with master...
I20260925 02:46:58.350608 32054 heartbeater.cc:511] Master 127.13.157.190:39467 requested a full tablet report, sending...
I20260925 02:46:58.351655 31732 ts_manager.cc:194] Registered new tserver with Master: d73cab0ba40e4c6c842faa1ebd990715 (127.13.157.130:36195)
I20260925 02:46:58.352417 31732 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.157.130:33273
W20260925 02:46:58.453850 32057 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:46:58.454094 32057 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:46:58.454130 32057 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:46:58.458042 32057 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:46:58.458094 32057 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:46:58.458145 32057 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.157.131
I20260925 02:46:58.462791 32057 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:36235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790304178325258-13942-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790304178325258-13942-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.157.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790304178325258-13942-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.157.131
--webserver_port=0
--tserver_master_addrs=127.13.157.190:39467
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32057
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.157.131
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790304178325258-13942-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:46:58.463909 32057 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:46:58.465268 32057 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 02:46:58.474516 32057 server_base.cc:1034] running on GCE node
W20260925 02:46:58.474478 32063 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:46:58.474468 32065 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:46:58.474483 32062 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:46:58.475133 32057 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:46:58.475768 32057 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:46:58.476950 32057 hybrid_clock.cc:648] HybridClock initialized: now 1790304418476908 us; error 51 us; skew 500 ppm
I20260925 02:46:58.478730 32057 webserver.cc:460] Webserver started at http://127.13.157.131:41127/ using document root <none> and password file <none>
I20260925 02:46:58.479332 32057 fs_manager.cc:359] Metadata directory not provided
I20260925 02:46:58.479399 32057 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:46:58.479598 32057 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:46:58.481581 32057 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790304178325258-13942-0/minicluster-data/ts-2/data/instance:
uuid: "075e6af311b34c7ca47f4cb6a859a613"
format_stamp: "Formatted at 2026-09-25 02:46:58 on dist-test-slave-kznw"
server_key: "62012167c8e81c36df8be1658bb3198b"
server_key_iv: "f66a05d4036f7f0ce4a3c2ec62d1f160"
server_key_version: "encryptionkey@0"
I20260925 02:46:58.482128 32057 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790304178325258-13942-0/minicluster-data/ts-2/wal/instance:
uuid: "075e6af311b34c7ca47f4cb6a859a613"
format_stamp: "Formatted at 2026-09-25 02:46:58 on dist-test-slave-kznw"
server_key: "62012167c8e81c36df8be1658bb3198b"
server_key_iv: "f66a05d4036f7f0ce4a3c2ec62d1f160"
server_key_version: "encryptionkey@0"
I20260925 02:46:58.485734 32057 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260925 02:46:58.488109 32071 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:46:58.489281 32057 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260925 02:46:58.489387 32057 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790304178325258-13942-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790304178325258-13942-0/minicluster-data/ts-2/wal
uuid: "075e6af311b34c7ca47f4cb6a859a613"
format_stamp: "Formatted at 2026-09-25 02:46:58 on dist-test-slave-kznw"
server_key: "62012167c8e81c36df8be1658bb3198b"
server_key_iv: "f66a05d4036f7f0ce4a3c2ec62d1f160"
server_key_version: "encryptionkey@0"
I20260925 02:46:58.489481 32057 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790304178325258-13942-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790304178325258-13942-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790304178325258-13942-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:46:58.506523 32057 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:46:58.507301 32057 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:46:58.507467 32057 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:46:58.508052 32057 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:46:58.508965 32057 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:46:58.509020 32057 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:46:58.509074 32057 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:46:58.509101 32057 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:46:58.517633 32057 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.131:39415
I20260925 02:46:58.517673 32184 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.131:39415 every 8 connection(s)
I20260925 02:46:58.518497 32057 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790304178325258-13942-0/minicluster-data/ts-2/data/info.pb
I20260925 02:46:58.522202 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 32057
I20260925 02:46:58.522293 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790304178325258-13942-0/minicluster-data/ts-2/wal/instance
I20260925 02:46:58.522557 13942 external_mini_cluster.cc:1428] Setting key 482b0b4de2c2361cf5a1cb4fa19933a1
I20260925 02:46:58.527491 31771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:58.520251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.131:49599 (local address 127.13.157.190:39467)
0925 02:46:58.520389 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0925 02:46:58.520393 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:46:58.521432 (+ 1039us) server_negotiation.cc:407] Connection header received
0925 02:46:58.522872 (+ 1440us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:46:58.522875 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:46:58.522951 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:46:58.523027 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:46:58.524064 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:46:58.524804 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:46:58.525875 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:46:58.526086 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:46:58.526154 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:46:58.527144 (+ 990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:46:58.527147 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:46:58.527149 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:46:58.527160 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:46:58.527178 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:46:58.527185 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:46:58.527237 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0925 02:46:58.527337 (+ 100us) server_negotiation.cc:299] Negotiation successful
0925 02:46:58.527383 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260925 02:46:58.528354 32187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:58.520644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:39467 (local address 127.13.157.131:49599)
0925 02:46:58.521287 (+ 643us) negotiation.cc:107] Waiting for socket to connect
0925 02:46:58.521327 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0925 02:46:58.522661 (+ 1334us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:46:58.523228 (+ 567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:46:58.523235 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:46:58.523342 (+ 107us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:46:58.523900 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:46:58.523920 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:46:58.524902 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:46:58.524905 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:46:58.525780 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:46:58.525790 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:46:58.526192 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:46:58.526194 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:46:58.526337 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:46:58.526872 (+ 535us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 02:46:58.526873 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:46:58.526947 (+ 74us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 02:46:58.527026 (+ 79us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:46:58.527245 (+ 219us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:46:58.527249 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:46:58.527252 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 02:46:58.527332 (+ 80us) client_negotiation.cc:240] Negotiation successful
0925 02:46:58.527601 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":106,"threads_started":1}
I20260925 02:46:58.529345 32185 heartbeater.cc:347] Connected to a master server at 127.13.157.190:39467
I20260925 02:46:58.529534 32185 heartbeater.cc:464] Registering TS with master...
I20260925 02:46:58.529973 32185 heartbeater.cc:511] Master 127.13.157.190:39467 requested a full tablet report, sending...
I20260925 02:46:58.530987 31732 ts_manager.cc:194] Registered new tserver with Master: 075e6af311b34c7ca47f4cb6a859a613 (127.13.157.131:39415)
I20260925 02:46:58.531539 31732 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.157.131:49599
I20260925 02:46:58.536405 13942 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 02:46:58.542001 31771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:58.538326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47830 (local address 127.13.157.190:39467)
0925 02:46:58.538546 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0925 02:46:58.538551 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:46:58.538642 (+ 91us) server_negotiation.cc:407] Connection header received
0925 02:46:58.538708 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:46:58.538710 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:46:58.538751 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:46:58.538812 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:46:58.539126 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:46:58.539696 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:46:58.540572 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:46:58.540795 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:46:58.540872 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:46:58.541626 (+ 754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:46:58.541629 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:46:58.541631 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:46:58.541642 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:46:58.541661 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:46:58.541668 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:46:58.541725 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0925 02:46:58.541797 (+ 72us) server_negotiation.cc:299] Negotiation successful
0925 02:46:58.541845 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125}
I20260925 02:46:58.544848 31732 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47830:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 02:46:58.547767 31732 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 02:46:58.562162 32198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:58.556487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.131:39415 (local address 127.0.0.1:57142)
0925 02:46:58.557457 (+ 970us) negotiation.cc:107] Waiting for socket to connect
0925 02:46:58.557467 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 02:46:58.557527 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:46:58.558115 (+ 588us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:46:58.558118 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:46:58.558135 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:46:58.558250 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:46:58.558255 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:46:58.559620 (+ 1365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:46:58.559623 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:46:58.560709 (+ 1086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:46:58.560717 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:46:58.561634 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:46:58.561638 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:46:58.561743 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:46:58.561785 (+ 42us) client_negotiation.cc:769] Sending connection context
0925 02:46:58.561893 (+ 108us) client_negotiation.cc:240] Negotiation successful
0925 02:46:58.562012 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":925,"mutex_wait_us":14,"thread_start_us":112,"threads_started":1}
I20260925 02:46:58.562621 32201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:58.557009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57142 (local address 127.13.157.131:39415)
0925 02:46:58.557727 (+ 718us) server_negotiation.cc:206] Beginning negotiation
0925 02:46:58.557736 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 02:46:58.557756 (+ 20us) server_negotiation.cc:407] Connection header received
0925 02:46:58.557828 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:46:58.557832 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:46:58.557972 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:46:58.558116 (+ 144us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:46:58.558403 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:46:58.559502 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:46:58.560836 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:46:58.561515 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:46:58.561613 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:46:58.561771 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0925 02:46:58.562353 (+ 582us) server_negotiation.cc:299] Negotiation successful
0925 02:46:58.562451 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":624,"thread_start_us":168,"threads_started":1}
I20260925 02:46:58.562637 32196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:58.556221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.129:40115 (local address 127.0.0.1:56876)
0925 02:46:58.556981 (+ 760us) negotiation.cc:107] Waiting for socket to connect
0925 02:46:58.557044 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0925 02:46:58.557182 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:46:58.558794 (+ 1612us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:46:58.558797 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:46:58.558816 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:46:58.558931 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:46:58.558938 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:46:58.560128 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:46:58.560131 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:46:58.561161 (+ 1030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:46:58.561172 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:46:58.562284 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:46:58.562288 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:46:58.562371 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:46:58.562393 (+ 22us) client_negotiation.cc:769] Sending connection context
0925 02:46:58.562458 (+ 65us) client_negotiation.cc:240] Negotiation successful
0925 02:46:58.562517 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":594,"mutex_wait_us":24,"spinlock_wait_cycles":5760,"thread_start_us":77,"threads_started":1}
I20260925 02:46:58.563035 32197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:58.556392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56876 (local address 127.13.157.129:40115)
0925 02:46:58.558385 (+ 1993us) server_negotiation.cc:206] Beginning negotiation
0925 02:46:58.558396 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 02:46:58.558417 (+ 21us) server_negotiation.cc:407] Connection header received
0925 02:46:58.558489 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:46:58.558494 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:46:58.558660 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:46:58.558797 (+ 137us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:46:58.559050 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:46:58.560004 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:46:58.561282 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:46:58.561932 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:46:58.562052 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:46:58.562216 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0925 02:46:58.562728 (+ 512us) server_negotiation.cc:299] Negotiation successful
0925 02:46:58.562833 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1891,"thread_start_us":112,"threads_started":1}
I20260925 02:46:58.563498 32199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:58.556470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.130:36195 (local address 127.0.0.1:40090)
0925 02:46:58.557013 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0925 02:46:58.557052 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0925 02:46:58.557183 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:46:58.558061 (+ 878us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:46:58.558067 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:46:58.558090 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:46:58.558220 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:46:58.558228 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:46:58.559338 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:46:58.559341 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:46:58.560697 (+ 1356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:46:58.560709 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:46:58.562886 (+ 2177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:46:58.562890 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:46:58.563220 (+ 330us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:46:58.563253 (+ 33us) client_negotiation.cc:769] Sending connection context
0925 02:46:58.563309 (+ 56us) client_negotiation.cc:240] Negotiation successful
0925 02:46:58.563373 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":479,"mutex_wait_us":23,"spinlock_wait_cycles":5760,"thread_start_us":115,"threads_started":1}
I20260925 02:46:58.564230 32200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:58.557022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40090 (local address 127.13.157.130:36195)
0925 02:46:58.557633 (+ 611us) server_negotiation.cc:206] Beginning negotiation
0925 02:46:58.557646 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0925 02:46:58.557670 (+ 24us) server_negotiation.cc:407] Connection header received
0925 02:46:58.557743 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:46:58.557748 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:46:58.557921 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:46:58.558039 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:46:58.558337 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:46:58.559247 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:46:58.561831 (+ 2584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:46:58.562694 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:46:58.562798 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:46:58.563943 (+ 1145us) server_negotiation.cc:1035] Waiting for connection context
0925 02:46:58.563990 (+ 47us) server_negotiation.cc:299] Negotiation successful
0925 02:46:58.564079 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":458,"thread_start_us":126,"threads_started":1}
I20260925 02:46:58.565723 31855 tablet_service.cc:1467] Processing CreateTablet for tablet cb37c72a98fa451da2016a1a8c380bab (DEFAULT_TABLE table=test-table [id=f9c542ba01d64e299f43c51ba8458aef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:46:58.565780 32119 tablet_service.cc:1467] Processing CreateTablet for tablet cb37c72a98fa451da2016a1a8c380bab (DEFAULT_TABLE table=test-table [id=f9c542ba01d64e299f43c51ba8458aef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:46:58.566656 31855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb37c72a98fa451da2016a1a8c380bab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:46:58.566761 32119 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb37c72a98fa451da2016a1a8c380bab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:46:58.566663 31988 tablet_service.cc:1467] Processing CreateTablet for tablet cb37c72a98fa451da2016a1a8c380bab (DEFAULT_TABLE table=test-table [id=f9c542ba01d64e299f43c51ba8458aef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:46:58.567582 31988 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb37c72a98fa451da2016a1a8c380bab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:46:58.572749 32202 tablet_bootstrap.cc:492] T cb37c72a98fa451da2016a1a8c380bab P 179db8bcab274aa58074cb94838a3136: Bootstrap starting.
I20260925 02:46:58.574283 32204 tablet_bootstrap.cc:492] T cb37c72a98fa451da2016a1a8c380bab P d73cab0ba40e4c6c842faa1ebd990715: Bootstrap starting.
I20260925 02:46:58.575103 32202 tablet_bootstrap.cc:654] T cb37c72a98fa451da2016a1a8c380bab P 179db8bcab274aa58074cb94838a3136: Neither blocks nor log segments found. Creating new log.
I20260925 02:46:58.575248 32203 tablet_bootstrap.cc:492] T cb37c72a98fa451da2016a1a8c380bab P 075e6af311b34c7ca47f4cb6a859a613: Bootstrap starting.
I20260925 02:46:58.575834 32202 log.cc:824] T cb37c72a98fa451da2016a1a8c380bab P 179db8bcab274aa58074cb94838a3136: Log is configured to *not* fsync() on all Append() calls
I20260925 02:46:58.576198 32204 tablet_bootstrap.cc:654] T cb37c72a98fa451da2016a1a8c380bab P d73cab0ba40e4c6c842faa1ebd990715: Neither blocks nor log segments found. Creating new log.
I20260925 02:46:58.576987 32204 log.cc:824] T cb37c72a98fa451da2016a1a8c380bab P d73cab0ba40e4c6c842faa1ebd990715: Log is configured to *not* fsync() on all Append() calls
I20260925 02:46:58.577167 32203 tablet_bootstrap.cc:654] T cb37c72a98fa451da2016a1a8c380bab P 075e6af311b34c7ca47f4cb6a859a613: Neither blocks nor log segments found. Creating new log.
I20260925 02:46:58.577828 32203 log.cc:824] T cb37c72a98fa451da2016a1a8c380bab P 075e6af311b34c7ca47f4cb6a859a613: Log is configured to *not* fsync() on all Append() calls
I20260925 02:46:58.578140 32202 tablet_bootstrap.cc:492] T cb37c72a98fa451da2016a1a8c380bab P 179db8bcab274aa58074cb94838a3136: No bootstrap required, opened a new log
I20260925 02:46:58.578317 32202 ts_tablet_manager.cc:1397] T cb37c72a98fa451da2016a1a8c380bab P 179db8bcab274aa58074cb94838a3136: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260925 02:46:58.579420 32203 tablet_bootstrap.cc:492] T cb37c72a98fa451da2016a1a8c380bab P 075e6af311b34c7ca47f4cb6a859a613: No bootstrap required, opened a new log
I20260925 02:46:58.579591 32203 ts_tablet_manager.cc:1397] T cb37c72a98fa451da2016a1a8c380bab P 075e6af311b34c7ca47f4cb6a859a613: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.001s
I20260925 02:46:58.579849 32204 tablet_bootstrap.cc:492] T cb37c72a98fa451da2016a1a8c380bab P d73cab0ba40e4c6c842faa1ebd990715: No bootstrap required, opened a new log
I20260925 02:46:58.580065 32204 ts_tablet_manager.cc:1397] T cb37c72a98fa451da2016a1a8c380bab P d73cab0ba40e4c6c842faa1ebd990715: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.003s
I20260925 02:46:58.582494 32203 raft_consensus.cc:348] T cb37c72a98fa451da2016a1a8c380bab P 075e6af311b34c7ca47f4cb6a859a613 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "179db8bcab274aa58074cb94838a3136" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 40115 } } peers { permanent_uuid: "d73cab0ba40e4c6c842faa1ebd990715" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 36195 } } peers { permanent_uuid: "075e6af311b34c7ca47f4cb6a859a613" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 39415 } }
I20260925 02:46:58.582719 32203 raft_consensus.cc:374] T cb37c72a98fa451da2016a1a8c380bab P 075e6af311b34c7ca47f4cb6a859a613 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:46:58.582541 32202 raft_consensus.cc:348] T cb37c72a98fa451da2016a1a8c380bab P 179db8bcab274aa58074cb94838a3136 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "179db8bcab274aa58074cb94838a3136" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 40115 } } peers { permanent_uuid: "d73cab0ba40e4c6c842faa1ebd990715" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 36195 } } peers { permanent_uuid: "075e6af311b34c7ca47f4cb6a859a613" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 39415 } }
I20260925 02:46:58.582769 32203 raft_consensus.cc:729] T cb37c72a98fa451da2016a1a8c380bab P 075e6af311b34c7ca47f4cb6a859a613 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 075e6af311b34c7ca47f4cb6a859a613, State: Initialized, Role: FOLLOWER
I20260925 02:46:58.582782 32202 raft_consensus.cc:374] T cb37c72a98fa451da2016a1a8c380bab P 179db8bcab274aa58074cb94838a3136 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:46:58.582845 32202 raft_consensus.cc:729] T cb37c72a98fa451da2016a1a8c380bab P 179db8bcab274aa58074cb94838a3136 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 179db8bcab274aa58074cb94838a3136, State: Initialized, Role: FOLLOWER
I20260925 02:46:58.583222 32203 consensus_queue.cc:260] T cb37c72a98fa451da2016a1a8c380bab P 075e6af311b34c7ca47f4cb6a859a613 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "179db8bcab274aa58074cb94838a3136" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 40115 } } peers { permanent_uuid: "d73cab0ba40e4c6c842faa1ebd990715" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 36195 } } peers { permanent_uuid: "075e6af311b34c7ca47f4cb6a859a613" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 39415 } }
I20260925 02:46:58.583310 32202 consensus_queue.cc:260] T cb37c72a98fa451da2016a1a8c380bab P 179db8bcab274aa58074cb94838a3136 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "179db8bcab274aa58074cb94838a3136" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 40115 } } peers { permanent_uuid: "d73cab0ba40e4c6c842faa1ebd990715" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 36195 } } peers { permanent_uuid: "075e6af311b34c7ca47f4cb6a859a613" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 39415 } }
I20260925 02:46:58.583813 32185 heartbeater.cc:503] Master 127.13.157.190:39467 was elected leader, sending a full tablet report...
I20260925 02:46:58.584168 32203 ts_tablet_manager.cc:1428] T cb37c72a98fa451da2016a1a8c380bab P 075e6af311b34c7ca47f4cb6a859a613: Time spent starting tablet: real 0.005s user 0.001s sys 0.004s
I20260925 02:46:58.584244 32202 ts_tablet_manager.cc:1428] T cb37c72a98fa451da2016a1a8c380bab P 179db8bcab274aa58074cb94838a3136: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260925 02:46:58.584762 32204 raft_consensus.cc:348] T cb37c72a98fa451da2016a1a8c380bab P d73cab0ba40e4c6c842faa1ebd990715 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "179db8bcab274aa58074cb94838a3136" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 40115 } } peers { permanent_uuid: "d73cab0ba40e4c6c842faa1ebd990715" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 36195 } } peers { permanent_uuid: "075e6af311b34c7ca47f4cb6a859a613" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 39415 } }
I20260925 02:46:58.585069 31923 heartbeater.cc:503] Master 127.13.157.190:39467 was elected leader, sending a full tablet report...
I20260925 02:46:58.585084 32204 raft_consensus.cc:374] T cb37c72a98fa451da2016a1a8c380bab P d73cab0ba40e4c6c842faa1ebd990715 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:46:58.585129 32204 raft_consensus.cc:729] T cb37c72a98fa451da2016a1a8c380bab P d73cab0ba40e4c6c842faa1ebd990715 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d73cab0ba40e4c6c842faa1ebd990715, State: Initialized, Role: FOLLOWER
I20260925 02:46:58.585516 32204 consensus_queue.cc:260] T cb37c72a98fa451da2016a1a8c380bab P d73cab0ba40e4c6c842faa1ebd990715 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "179db8bcab274aa58074cb94838a3136" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 40115 } } peers { permanent_uuid: "d73cab0ba40e4c6c842faa1ebd990715" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 36195 } } peers { permanent_uuid: "075e6af311b34c7ca47f4cb6a859a613" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 39415 } }
I20260925 02:46:58.586418 32204 ts_tablet_manager.cc:1428] T cb37c72a98fa451da2016a1a8c380bab P d73cab0ba40e4c6c842faa1ebd990715: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260925 02:46:58.586603 32054 heartbeater.cc:503] Master 127.13.157.190:39467 was elected leader, sending a full tablet report...
W20260925 02:46:58.588989 32055 tablet.cc:2369] T cb37c72a98fa451da2016a1a8c380bab P d73cab0ba40e4c6c842faa1ebd990715: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 02:46:58.618602 32208 raft_consensus.cc:482] T cb37c72a98fa451da2016a1a8c380bab P 075e6af311b34c7ca47f4cb6a859a613 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 02:46:58.618813 32208 raft_consensus.cc:504] T cb37c72a98fa451da2016a1a8c380bab P 075e6af311b34c7ca47f4cb6a859a613 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "179db8bcab274aa58074cb94838a3136" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 40115 } } peers { permanent_uuid: "d73cab0ba40e4c6c842faa1ebd990715" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 36195 } } peers { permanent_uuid: "075e6af311b34c7ca47f4cb6a859a613" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 39415 } }
I20260925 02:46:58.620116 32208 leader_election.cc:288] T cb37c72a98fa451da2016a1a8c380bab P 075e6af311b34c7ca47f4cb6a859a613 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 179db8bcab274aa58074cb94838a3136 (127.13.157.129:40115), d73cab0ba40e4c6c842faa1ebd990715 (127.13.157.130:36195)
I20260925 02:46:58.623698 32197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:58.620432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.131:43445 (local address 127.13.157.129:40115)
0925 02:46:58.620549 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0925 02:46:58.620554 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:46:58.620574 (+ 20us) server_negotiation.cc:407] Connection header received
0925 02:46:58.620765 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:46:58.620767 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:46:58.620815 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:46:58.620890 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:46:58.621170 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:46:58.621675 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:46:58.622688 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:46:58.623149 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:46:58.623263 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:46:58.623304 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0925 02:46:58.623513 (+ 209us) server_negotiation.cc:299] Negotiation successful
0925 02:46:58.623567 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260925 02:46:58.623723 32187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:58.620326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.129:40115 (local address 127.13.157.131:43445)
0925 02:46:58.620519 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0925 02:46:58.620535 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 02:46:58.620632 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:46:58.620915 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:46:58.620918 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:46:58.620937 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:46:58.621051 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:46:58.621058 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:46:58.621784 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:46:58.621787 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:46:58.622563 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:46:58.622572 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:46:58.623271 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:46:58.623275 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:46:58.623383 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:46:58.623413 (+ 30us) client_negotiation.cc:769] Sending connection context
0925 02:46:58.623504 (+ 91us) client_negotiation.cc:240] Negotiation successful
0925 02:46:58.623581 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114}
I20260925 02:46:58.624051 32200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:58.620477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.131:34649 (local address 127.13.157.130:36195)
0925 02:46:58.620619 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0925 02:46:58.620624 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:46:58.620873 (+ 249us) server_negotiation.cc:407] Connection header received
0925 02:46:58.620996 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:46:58.620999 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:46:58.621046 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:46:58.621114 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:46:58.621410 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:46:58.621913 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:46:58.622946 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:46:58.623518 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:46:58.623642 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:46:58.623690 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0925 02:46:58.623886 (+ 196us) server_negotiation.cc:299] Negotiation successful
0925 02:46:58.623958 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260925 02:46:58.624382 32211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:58.620392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.130:36195 (local address 127.13.157.131:34649)
0925 02:46:58.620780 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0925 02:46:58.620792 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 02:46:58.620895 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:46:58.621132 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:46:58.621135 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:46:58.621151 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:46:58.621275 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:46:58.621281 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:46:58.622022 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:46:58.622026 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:46:58.622787 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:46:58.622796 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:46:58.623635 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:46:58.623639 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:46:58.623732 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:46:58.623755 (+ 23us) client_negotiation.cc:769] Sending connection context
0925 02:46:58.624178 (+ 423us) client_negotiation.cc:240] Negotiation successful
0925 02:46:58.624250 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":165,"threads_started":1}
I20260925 02:46:58.624578 31876 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb37c72a98fa451da2016a1a8c380bab" candidate_uuid: "075e6af311b34c7ca47f4cb6a859a613" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "179db8bcab274aa58074cb94838a3136" is_pre_election: true
I20260925 02:46:58.624900 31876 raft_consensus.cc:2454] T cb37c72a98fa451da2016a1a8c380bab P 179db8bcab274aa58074cb94838a3136 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 075e6af311b34c7ca47f4cb6a859a613 in term 0.
I20260925 02:46:58.624960 32008 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb37c72a98fa451da2016a1a8c380bab" candidate_uuid: "075e6af311b34c7ca47f4cb6a859a613" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d73cab0ba40e4c6c842faa1ebd990715" is_pre_election: true
I20260925 02:46:58.625231 32008 raft_consensus.cc:2454] T cb37c72a98fa451da2016a1a8c380bab P d73cab0ba40e4c6c842faa1ebd990715 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 075e6af311b34c7ca47f4cb6a859a613 in term 0.
I20260925 02:46:58.625379 32075 leader_election.cc:302] T cb37c72a98fa451da2016a1a8c380bab P 075e6af311b34c7ca47f4cb6a859a613 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 075e6af311b34c7ca47f4cb6a859a613, 179db8bcab274aa58074cb94838a3136; no voters:
I20260925 02:46:58.625648 32208 raft_consensus.cc:2781] T cb37c72a98fa451da2016a1a8c380bab P 075e6af311b34c7ca47f4cb6a859a613 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 02:46:58.625742 32208 raft_consensus.cc:482] T cb37c72a98fa451da2016a1a8c380bab P 075e6af311b34c7ca47f4cb6a859a613 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 02:46:58.625800 32208 raft_consensus.cc:3037] T cb37c72a98fa451da2016a1a8c380bab P 075e6af311b34c7ca47f4cb6a859a613 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:46:58.626825 32208 raft_consensus.cc:504] T cb37c72a98fa451da2016a1a8c380bab P 075e6af311b34c7ca47f4cb6a859a613 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "179db8bcab274aa58074cb94838a3136" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 40115 } } peers { permanent_uuid: "d73cab0ba40e4c6c842faa1ebd990715" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 36195 } } peers { permanent_uuid: "075e6af311b34c7ca47f4cb6a859a613" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 39415 } }
I20260925 02:46:58.627197 32208 leader_election.cc:288] T cb37c72a98fa451da2016a1a8c380bab P 075e6af311b34c7ca47f4cb6a859a613 [CANDIDATE]: Term 1 election: Requested vote from peers 179db8bcab274aa58074cb94838a3136 (127.13.157.129:40115), d73cab0ba40e4c6c842faa1ebd990715 (127.13.157.130:36195)
I20260925 02:46:58.627516 31876 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb37c72a98fa451da2016a1a8c380bab" candidate_uuid: "075e6af311b34c7ca47f4cb6a859a613" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "179db8bcab274aa58074cb94838a3136"
I20260925 02:46:58.627527 32008 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb37c72a98fa451da2016a1a8c380bab" candidate_uuid: "075e6af311b34c7ca47f4cb6a859a613" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d73cab0ba40e4c6c842faa1ebd990715"
I20260925 02:46:58.627630 32008 raft_consensus.cc:3037] T cb37c72a98fa451da2016a1a8c380bab P d73cab0ba40e4c6c842faa1ebd990715 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:46:58.627645 31876 raft_consensus.cc:3037] T cb37c72a98fa451da2016a1a8c380bab P 179db8bcab274aa58074cb94838a3136 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:46:58.628671 32008 raft_consensus.cc:2454] T cb37c72a98fa451da2016a1a8c380bab P d73cab0ba40e4c6c842faa1ebd990715 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 075e6af311b34c7ca47f4cb6a859a613 in term 1.
I20260925 02:46:58.628697 31876 raft_consensus.cc:2454] T cb37c72a98fa451da2016a1a8c380bab P 179db8bcab274aa58074cb94838a3136 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 075e6af311b34c7ca47f4cb6a859a613 in term 1.
I20260925 02:46:58.628983 32074 leader_election.cc:302] T cb37c72a98fa451da2016a1a8c380bab P 075e6af311b34c7ca47f4cb6a859a613 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 075e6af311b34c7ca47f4cb6a859a613, d73cab0ba40e4c6c842faa1ebd990715; no voters:
I20260925 02:46:58.629172 32208 raft_consensus.cc:2781] T cb37c72a98fa451da2016a1a8c380bab P 075e6af311b34c7ca47f4cb6a859a613 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:46:58.629458 32208 raft_consensus.cc:686] T cb37c72a98fa451da2016a1a8c380bab P 075e6af311b34c7ca47f4cb6a859a613 [term 1 LEADER]: Becoming Leader. State: Replica: 075e6af311b34c7ca47f4cb6a859a613, State: Running, Role: LEADER
I20260925 02:46:58.629791 32208 consensus_queue.cc:237] T cb37c72a98fa451da2016a1a8c380bab P 075e6af311b34c7ca47f4cb6a859a613 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "179db8bcab274aa58074cb94838a3136" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 40115 } } peers { permanent_uuid: "d73cab0ba40e4c6c842faa1ebd990715" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 36195 } } peers { permanent_uuid: "075e6af311b34c7ca47f4cb6a859a613" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 39415 } }
I20260925 02:46:58.632810 31731 catalog_manager.cc:5495] T cb37c72a98fa451da2016a1a8c380bab P 075e6af311b34c7ca47f4cb6a859a613 reported cstate change: term changed from 0 to 1, leader changed from <none> to 075e6af311b34c7ca47f4cb6a859a613 (127.13.157.131). New cstate: current_term: 1 leader_uuid: "075e6af311b34c7ca47f4cb6a859a613" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "179db8bcab274aa58074cb94838a3136" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 40115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d73cab0ba40e4c6c842faa1ebd990715" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 36195 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "075e6af311b34c7ca47f4cb6a859a613" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 39415 } health_report { overall_health: HEALTHY } } }
W20260925 02:46:58.642748 31924 tablet.cc:2369] T cb37c72a98fa451da2016a1a8c380bab P 179db8bcab274aa58074cb94838a3136: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 02:46:58.658744 32201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:58.655466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57152 (local address 127.13.157.131:39415)
0925 02:46:58.655616 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0925 02:46:58.655619 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 02:46:58.655634 (+ 15us) server_negotiation.cc:407] Connection header received
0925 02:46:58.655710 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:46:58.655712 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:46:58.655763 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:46:58.655848 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 02:46:58.656014 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:46:58.656487 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:46:58.657341 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:46:58.657629 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:46:58.657728 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:46:58.657965 (+ 237us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 02:46:58.658395 (+ 430us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 02:46:58.658495 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0925 02:46:58.658553 (+ 58us) server_negotiation.cc:299] Negotiation successful
0925 02:46:58.658606 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260925 02:46:58.664764 32008 raft_consensus.cc:1261] T cb37c72a98fa451da2016a1a8c380bab P d73cab0ba40e4c6c842faa1ebd990715 [term 1 FOLLOWER]: Refusing update from remote peer 075e6af311b34c7ca47f4cb6a859a613: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:46:58.664770 31876 raft_consensus.cc:1261] T cb37c72a98fa451da2016a1a8c380bab P 179db8bcab274aa58074cb94838a3136 [term 1 FOLLOWER]: Refusing update from remote peer 075e6af311b34c7ca47f4cb6a859a613: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:46:58.665401 32208 consensus_queue.cc:1035] T cb37c72a98fa451da2016a1a8c380bab P 075e6af311b34c7ca47f4cb6a859a613 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d73cab0ba40e4c6c842faa1ebd990715" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 36195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:46:58.665586 32212 consensus_queue.cc:1035] T cb37c72a98fa451da2016a1a8c380bab P 075e6af311b34c7ca47f4cb6a859a613 [LEADER]: Connected to new peer: Peer: permanent_uuid: "179db8bcab274aa58074cb94838a3136" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 40115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:46:58.673571 32217 mvcc.cc:205] Tried to move back new op lower bound from 7333086898843987968 to 7333086898710315008. Current Snapshot: MvccSnapshot[applied={T|T < 7333086898843987968}]
I20260925 02:46:58.681836 31731 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47830:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 02:46:58.681975 31731 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47830:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 02:46:58.684288 31731 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ea93940d6a304708817719383144178d: Sending DeleteTablet for 3 replicas of tablet cb37c72a98fa451da2016a1a8c380bab
I20260925 02:46:58.684988 31855 tablet_service.cc:1514] Processing DeleteTablet for tablet cb37c72a98fa451da2016a1a8c380bab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:46:58 UTC) from {username='slave'} at 127.0.0.1:56876
I20260925 02:46:58.685012 32119 tablet_service.cc:1514] Processing DeleteTablet for tablet cb37c72a98fa451da2016a1a8c380bab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:46:58 UTC) from {username='slave'} at 127.0.0.1:57142
I20260925 02:46:58.685060 31988 tablet_service.cc:1514] Processing DeleteTablet for tablet cb37c72a98fa451da2016a1a8c380bab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:46:58 UTC) from {username='slave'} at 127.0.0.1:40090
I20260925 02:46:58.685505 32225 tablet_replica.cc:330] T cb37c72a98fa451da2016a1a8c380bab P d73cab0ba40e4c6c842faa1ebd990715: stopping tablet replica
I20260925 02:46:58.685729 32227 tablet_replica.cc:330] T cb37c72a98fa451da2016a1a8c380bab P 075e6af311b34c7ca47f4cb6a859a613: stopping tablet replica
I20260925 02:46:58.685873 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 31793
I20260925 02:46:58.686080 32227 raft_consensus.cc:2229] T cb37c72a98fa451da2016a1a8c380bab P 075e6af311b34c7ca47f4cb6a859a613 [term 1 LEADER]: Raft consensus shutting down.
I20260925 02:46:58.686481 32227 raft_consensus.cc:2258] T cb37c72a98fa451da2016a1a8c380bab P 075e6af311b34c7ca47f4cb6a859a613 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:46:58.686957 32225 raft_consensus.cc:2229] T cb37c72a98fa451da2016a1a8c380bab P d73cab0ba40e4c6c842faa1ebd990715 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 02:46:58.687255 32225 raft_consensus.cc:2258] T cb37c72a98fa451da2016a1a8c380bab P d73cab0ba40e4c6c842faa1ebd990715 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:46:58.689633 32225 ts_tablet_manager.cc:1905] T cb37c72a98fa451da2016a1a8c380bab P d73cab0ba40e4c6c842faa1ebd990715: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:46:58.692106 32227 ts_tablet_manager.cc:1905] T cb37c72a98fa451da2016a1a8c380bab P 075e6af311b34c7ca47f4cb6a859a613: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:46:58.692281 32225 ts_tablet_manager.cc:1918] T cb37c72a98fa451da2016a1a8c380bab P d73cab0ba40e4c6c842faa1ebd990715: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 02:46:58.692371 32225 log.cc:1196] T cb37c72a98fa451da2016a1a8c380bab P d73cab0ba40e4c6c842faa1ebd990715: Deleting WAL directory at /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790304178325258-13942-0/minicluster-data/ts-1/wal/wals/cb37c72a98fa451da2016a1a8c380bab
I20260925 02:46:58.692633 32225 ts_tablet_manager.cc:1939] T cb37c72a98fa451da2016a1a8c380bab P d73cab0ba40e4c6c842faa1ebd990715: Deleting consensus metadata
I20260925 02:46:58.693868 31718 catalog_manager.cc:4841] TS d73cab0ba40e4c6c842faa1ebd990715 (127.13.157.130:36195): tablet cb37c72a98fa451da2016a1a8c380bab (table test-table [id=f9c542ba01d64e299f43c51ba8458aef]) successfully deleted
W20260925 02:46:58.694176 31719 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.157.129:40115 (error 108)
I20260925 02:46:58.694408 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 31926
I20260925 02:46:58.694849 32227 ts_tablet_manager.cc:1918] T cb37c72a98fa451da2016a1a8c380bab P 075e6af311b34c7ca47f4cb6a859a613: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 02:46:58.694957 32227 log.cc:1196] T cb37c72a98fa451da2016a1a8c380bab P 075e6af311b34c7ca47f4cb6a859a613: Deleting WAL directory at /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790304178325258-13942-0/minicluster-data/ts-2/wal/wals/cb37c72a98fa451da2016a1a8c380bab
I20260925 02:46:58.695240 32227 ts_tablet_manager.cc:1939] T cb37c72a98fa451da2016a1a8c380bab P 075e6af311b34c7ca47f4cb6a859a613: Deleting consensus metadata
I20260925 02:46:58.696303 31716 catalog_manager.cc:4841] TS 075e6af311b34c7ca47f4cb6a859a613 (127.13.157.131:39415): tablet cb37c72a98fa451da2016a1a8c380bab (table test-table [id=f9c542ba01d64e299f43c51ba8458aef]) successfully deleted
I20260925 02:46:58.696868 32196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:58.696600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.129:40115 (local address 127.0.0.1:56888)
0925 02:46:58.696716 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0925 02:46:58.696786 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.129:40115: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260925 02:46:58.697008 31719 catalog_manager.cc:4584] TS 179db8bcab274aa58074cb94838a3136 (127.13.157.129:40115): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet cb37c72a98fa451da2016a1a8c380bab: Network error: Client connection negotiation failed: client connection to 127.13.157.129:40115: connect: Connection refused (error 111)
I20260925 02:46:58.700582 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 32057
I20260925 02:46:58.705457 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 31701
2026-09-25T02:46:58Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1015 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790304178325258-13942-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 02:46:58 dist-test-slave-kznw krb5kdc[32231](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 02:46:58 dist-test-slave-kznw krb5kdc[32231](info): set up 2 sockets
Sep 25 02:46:58 dist-test-slave-kznw krb5kdc[32231](info): commencing operation
krb5kdc: starting...
W20260925 02:46:59.662380 13942 mini_kdc.cc:121] Time spent starting KDC: real 0.937s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790304178325258-13942-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790304178325258-13942-0/krb5kdc/test-user.keytab.
Sep 25 02:46:59 dist-test-slave-kznw krb5kdc[32231](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304419, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T02:46:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T02:46:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.157.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.157.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.157.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab.
Entry for principal kudu/127.13.157.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab.
I20260925 02:46:59.782151 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790304178325258-13942-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790304178325258-13942-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790304178325258-13942-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.157.190:42389
--webserver_interface=127.13.157.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab
--principal=kudu/127.13.157.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:41155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.157.190:42389
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:46:59.891101 32244 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:46:59.891371 32244 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:46:59.891429 32244 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:46:59.895453 32244 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:46:59.895524 32244 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:46:59.895550 32244 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:46:59.895572 32244 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:46:59.895601 32244 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:46:59.901026 32244 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:41155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790304178325258-13942-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790304178325258-13942-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.157.190:42389
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab
--principal=kudu/127.13.157.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.157.190:42389
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.157.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32244
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790304178325258-13942-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:46:59.902124 32244 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:46:59.903196 32244 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:46:59.911399 32249 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:46:59.911389 32252 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:46:59.911602 32244 server_base.cc:1034] running on GCE node
W20260925 02:46:59.911389 32250 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:46:59.912276 32244 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:46:59.913347 32244 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:46:59.914543 32244 hybrid_clock.cc:648] HybridClock initialized: now 1790304419914533 us; error 43 us; skew 500 ppm
Sep 25 02:46:59 dist-test-slave-kznw krb5kdc[32231](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304419, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:46:59.917081 32244 init.cc:377] Logged in from keytab as kudu/127.13.157.190@KRBTEST.COM (short username kudu)
I20260925 02:46:59.917945 32244 webserver.cc:460] Webserver started at http://127.13.157.190:40833/ using document root <none> and password file <none>
I20260925 02:46:59.918458 32244 fs_manager.cc:359] Metadata directory not provided
I20260925 02:46:59.918532 32244 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:46:59.918730 32244 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:46:59.920766 32244 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790304178325258-13942-0/minicluster-data/master-0/data/instance:
uuid: "c65e2f3f88d149928e6946de48948cfd"
format_stamp: "Formatted at 2026-09-25 02:46:59 on dist-test-slave-kznw"
server_key: "1ae1e7f7ad59c1fffcef0992c27ce608"
server_key_iv: "f10e9a82a1c2e246dc3ffd8dc8d93541"
server_key_version: "encryptionkey@0"
I20260925 02:46:59.921293 32244 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790304178325258-13942-0/minicluster-data/master-0/wal/instance:
uuid: "c65e2f3f88d149928e6946de48948cfd"
format_stamp: "Formatted at 2026-09-25 02:46:59 on dist-test-slave-kznw"
server_key: "1ae1e7f7ad59c1fffcef0992c27ce608"
server_key_iv: "f10e9a82a1c2e246dc3ffd8dc8d93541"
server_key_version: "encryptionkey@0"
I20260925 02:46:59.924961 32244 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260925 02:46:59.927376 32259 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:46:59.928588 32244 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260925 02:46:59.928714 32244 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790304178325258-13942-0/minicluster-data/master-0/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790304178325258-13942-0/minicluster-data/master-0/wal
uuid: "c65e2f3f88d149928e6946de48948cfd"
format_stamp: "Formatted at 2026-09-25 02:46:59 on dist-test-slave-kznw"
server_key: "1ae1e7f7ad59c1fffcef0992c27ce608"
server_key_iv: "f10e9a82a1c2e246dc3ffd8dc8d93541"
server_key_version: "encryptionkey@0"
I20260925 02:46:59.928810 32244 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790304178325258-13942-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790304178325258-13942-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790304178325258-13942-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:46:59.945940 32244 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:46:59.948902 32244 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:46:59.949080 32244 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:46:59.955649 32244 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.190:42389
I20260925 02:46:59.955699 32311 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.190:42389 every 8 connection(s)
I20260925 02:46:59.956574 32244 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
I20260925 02:46:59.957859 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 32244
I20260925 02:46:59.958015 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790304178325258-13942-0/minicluster-data/master-0/wal/instance
I20260925 02:46:59.958307 13942 external_mini_cluster.cc:1428] Setting key 30cbcddd8773ebd5d6c523b8e856cc22
I20260925 02:46:59.959568 32312 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:46:59.966351 32312 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c65e2f3f88d149928e6946de48948cfd: Bootstrap starting.
Sep 25 02:46:59 dist-test-slave-kznw krb5kdc[32231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304419, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.157.190@KRBTEST.COM
I20260925 02:46:59.968971 32312 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c65e2f3f88d149928e6946de48948cfd: Neither blocks nor log segments found. Creating new log.
I20260925 02:46:59.969791 32312 log.cc:824] T 00000000000000000000000000000000 P c65e2f3f88d149928e6946de48948cfd: Log is configured to *not* fsync() on all Append() calls
I20260925 02:46:59.971987 32312 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c65e2f3f88d149928e6946de48948cfd: No bootstrap required, opened a new log
I20260925 02:46:59.974401 32315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:46:59.959766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46552 (local address 127.13.157.190:42389)
0925 02:46:59.960184 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0925 02:46:59.960202 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0925 02:46:59.960244 (+ 42us) server_negotiation.cc:407] Connection header received
0925 02:46:59.961055 (+ 811us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:46:59.961078 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:46:59.961485 (+ 407us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:46:59.961842 (+ 357us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:46:59.962504 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:46:59.964014 (+ 1510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:46:59.964982 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:46:59.965693 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:46:59.965860 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:46:59.968981 (+ 3121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:46:59.968988 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:46:59.969007 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:46:59.969049 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:46:59.971108 (+ 2059us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:46:59.971610 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:46:59.971615 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:46:59.971621 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:46:59.971748 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:46:59.972121 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:46:59.972125 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:46:59.972126 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:46:59.972642 (+ 516us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:46:59.972827 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0925 02:46:59.973096 (+ 269us) server_negotiation.cc:299] Negotiation successful
0925 02:46:59.973399 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":274,"thread_start_us":123,"threads_started":1}
I20260925 02:46:59.976356 32312 raft_consensus.cc:348] T 00000000000000000000000000000000 P c65e2f3f88d149928e6946de48948cfd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c65e2f3f88d149928e6946de48948cfd" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 42389 } }
I20260925 02:46:59.976634 32312 raft_consensus.cc:374] T 00000000000000000000000000000000 P c65e2f3f88d149928e6946de48948cfd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:46:59.976747 32312 raft_consensus.cc:729] T 00000000000000000000000000000000 P c65e2f3f88d149928e6946de48948cfd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c65e2f3f88d149928e6946de48948cfd, State: Initialized, Role: FOLLOWER
I20260925 02:46:59.977137 32312 consensus_queue.cc:260] T 00000000000000000000000000000000 P c65e2f3f88d149928e6946de48948cfd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c65e2f3f88d149928e6946de48948cfd" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 42389 } }
I20260925 02:46:59.977260 32312 raft_consensus.cc:388] T 00000000000000000000000000000000 P c65e2f3f88d149928e6946de48948cfd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 02:46:59.977305 32312 raft_consensus.cc:482] T 00000000000000000000000000000000 P c65e2f3f88d149928e6946de48948cfd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 02:46:59.977375 32312 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c65e2f3f88d149928e6946de48948cfd [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:46:59.978207 32312 raft_consensus.cc:504] T 00000000000000000000000000000000 P c65e2f3f88d149928e6946de48948cfd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c65e2f3f88d149928e6946de48948cfd" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 42389 } }
I20260925 02:46:59.978518 32312 leader_election.cc:302] T 00000000000000000000000000000000 P c65e2f3f88d149928e6946de48948cfd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c65e2f3f88d149928e6946de48948cfd; no voters:
I20260925 02:46:59.978785 32312 leader_election.cc:288] T 00000000000000000000000000000000 P c65e2f3f88d149928e6946de48948cfd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 02:46:59.978951 32317 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c65e2f3f88d149928e6946de48948cfd [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:46:59.979187 32317 raft_consensus.cc:686] T 00000000000000000000000000000000 P c65e2f3f88d149928e6946de48948cfd [term 1 LEADER]: Becoming Leader. State: Replica: c65e2f3f88d149928e6946de48948cfd, State: Running, Role: LEADER
I20260925 02:46:59.979537 32317 consensus_queue.cc:237] T 00000000000000000000000000000000 P c65e2f3f88d149928e6946de48948cfd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c65e2f3f88d149928e6946de48948cfd" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 42389 } }
I20260925 02:46:59.979981 32312 sys_catalog.cc:564] T 00000000000000000000000000000000 P c65e2f3f88d149928e6946de48948cfd [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:46:59.981314 32318 sys_catalog.cc:455] T 00000000000000000000000000000000 P c65e2f3f88d149928e6946de48948cfd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c65e2f3f88d149928e6946de48948cfd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c65e2f3f88d149928e6946de48948cfd" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 42389 } } }
I20260925 02:46:59.981345 32319 sys_catalog.cc:455] T 00000000000000000000000000000000 P c65e2f3f88d149928e6946de48948cfd [sys.catalog]: SysCatalogTable state changed. Reason: New leader c65e2f3f88d149928e6946de48948cfd. Latest consensus state: current_term: 1 leader_uuid: "c65e2f3f88d149928e6946de48948cfd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c65e2f3f88d149928e6946de48948cfd" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 42389 } } }
I20260925 02:46:59.981441 32318 sys_catalog.cc:458] T 00000000000000000000000000000000 P c65e2f3f88d149928e6946de48948cfd [sys.catalog]: This master's current role is: LEADER
I20260925 02:46:59.981465 32319 sys_catalog.cc:458] T 00000000000000000000000000000000 P c65e2f3f88d149928e6946de48948cfd [sys.catalog]: This master's current role is: LEADER
I20260925 02:46:59.981786 32327 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:46:59.984527 32327 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:46:59.989549 32327 catalog_manager.cc:1347] Generated new cluster ID: 8c70b1b15b364c1eaa254845b151a938
I20260925 02:46:59.989621 32327 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 02:47:00.003175 32327 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 02:47:00.004174 32327 catalog_manager.cc:1504] Loading token signing keys...
I20260925 02:47:00.014729 32327 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c65e2f3f88d149928e6946de48948cfd: Generated new TSK 0
I20260925 02:47:00.015641 32327 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.157.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.157.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.157.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.129.keytab.
Entry for principal kudu/127.13.157.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.129.keytab.
I20260925 02:47:00.046242 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790304178325258-13942-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790304178325258-13942-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790304178325258-13942-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790304178325258-13942-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.157.129:0
--local_ip_for_outbound_sockets=127.13.157.129
--webserver_interface=127.13.157.129
--webserver_port=0
--tserver_master_addrs=127.13.157.190:42389
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.129.keytab
--principal=kudu/127.13.157.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:41155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:47:00.158102 32338 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:47:00.158378 32338 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:47:00.158422 32338 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:47:00.162525 32338 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:47:00.162590 32338 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:47:00.162657 32338 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.157.129
I20260925 02:47:00.168008 32338 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:41155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790304178325258-13942-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790304178325258-13942-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.129.keytab
--principal=kudu/127.13.157.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.157.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790304178325258-13942-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.157.129
--webserver_port=0
--tserver_master_addrs=127.13.157.190:42389
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32338
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.157.129
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790304178325258-13942-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:47:00.169113 32338 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:47:00.170158 32338 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:47:00.179553 32344 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:47:00.179657 32338 server_base.cc:1034] running on GCE node
W20260925 02:47:00.179595 32343 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:47:00.179553 32346 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:47:00.180267 32338 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:47:00.180879 32338 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:47:00.182032 32338 hybrid_clock.cc:648] HybridClock initialized: now 1790304420181975 us; error 68 us; skew 500 ppm
Sep 25 02:47:00 dist-test-slave-kznw krb5kdc[32231](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304420, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:47:00.184612 32338 init.cc:377] Logged in from keytab as kudu/127.13.157.129@KRBTEST.COM (short username kudu)
I20260925 02:47:00.185438 32338 webserver.cc:460] Webserver started at http://127.13.157.129:39167/ using document root <none> and password file <none>
I20260925 02:47:00.185974 32338 fs_manager.cc:359] Metadata directory not provided
I20260925 02:47:00.186041 32338 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:47:00.186275 32338 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:47:00.188313 32338 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790304178325258-13942-0/minicluster-data/ts-0/data/instance:
uuid: "3287457e810f47ecbc499d5a29fcfffd"
format_stamp: "Formatted at 2026-09-25 02:47:00 on dist-test-slave-kznw"
server_key: "10d0a5c65c67bed44a055e1c5c029bc0"
server_key_iv: "b5c9e08b314227de206d1171acd0b31c"
server_key_version: "encryptionkey@0"
I20260925 02:47:00.188848 32338 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790304178325258-13942-0/minicluster-data/ts-0/wal/instance:
uuid: "3287457e810f47ecbc499d5a29fcfffd"
format_stamp: "Formatted at 2026-09-25 02:47:00 on dist-test-slave-kznw"
server_key: "10d0a5c65c67bed44a055e1c5c029bc0"
server_key_iv: "b5c9e08b314227de206d1171acd0b31c"
server_key_version: "encryptionkey@0"
I20260925 02:47:00.192508 32338 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260925 02:47:00.194880 32353 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:47:00.196307 32338 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260925 02:47:00.196416 32338 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790304178325258-13942-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790304178325258-13942-0/minicluster-data/ts-0/wal
uuid: "3287457e810f47ecbc499d5a29fcfffd"
format_stamp: "Formatted at 2026-09-25 02:47:00 on dist-test-slave-kznw"
server_key: "10d0a5c65c67bed44a055e1c5c029bc0"
server_key_iv: "b5c9e08b314227de206d1171acd0b31c"
server_key_version: "encryptionkey@0"
I20260925 02:47:00.196523 32338 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790304178325258-13942-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790304178325258-13942-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790304178325258-13942-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:47:00.211630 32338 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:47:00.214613 32338 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:47:00.214790 32338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:47:00.215409 32338 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:47:00.216252 32338 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:47:00.216298 32338 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:47:00.216351 32338 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:47:00.216379 32338 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:47:00.225150 32338 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.129:44245
I20260925 02:47:00.225175 32466 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.129:44245 every 8 connection(s)
I20260925 02:47:00.226042 32338 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790304178325258-13942-0/minicluster-data/ts-0/data/info.pb
I20260925 02:47:00.232720 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 32338
I20260925 02:47:00.232856 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790304178325258-13942-0/minicluster-data/ts-0/wal/instance
I20260925 02:47:00.233141 13942 external_mini_cluster.cc:1428] Setting key 3afa8fec764d94fe602f74367628b1ea
Sep 25 02:47:00 dist-test-slave-kznw krb5kdc[32231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304420, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.129@KRBTEST.COM for kudu/127.13.157.190@KRBTEST.COM
I20260925 02:47:00.240324 32315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:00.227900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.129:50003 (local address 127.13.157.190:42389)
0925 02:47:00.228074 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:00.228080 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:00.228929 (+ 849us) server_negotiation.cc:407] Connection header received
0925 02:47:00.230240 (+ 1311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:00.230242 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:00.230329 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:00.230410 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:47:00.231621 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:00.232258 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:00.233699 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:00.233950 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:00.234046 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:00.236839 (+ 2793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:47:00.236842 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:47:00.236844 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:47:00.236869 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:47:00.238225 (+ 1356us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:47:00.238784 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:47:00.238787 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:47:00.238788 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:47:00.238866 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:47:00.239214 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:47:00.239217 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:47:00.239219 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:47:00.239411 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:47:00.239498 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:00.240015 (+ 517us) server_negotiation.cc:299] Negotiation successful
0925 02:47:00.240154 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260925 02:47:00.241153 32469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:00.228302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:42389 (local address 127.13.157.129:50003)
0925 02:47:00.228780 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0925 02:47:00.228823 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0925 02:47:00.230027 (+ 1204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:47:00.230586 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:47:00.230593 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:47:00.230965 (+ 372us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:47:00.231470 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:47:00.231484 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:47:00.232374 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:47:00.232377 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:47:00.233518 (+ 1141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:47:00.233531 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:47:00.234107 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:47:00.234111 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:47:00.234301 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:00.234990 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:47:00.235006 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:47:00.236676 (+ 1670us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:47:00.238337 (+ 1661us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:47:00.238343 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:47:00.238360 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:47:00.238691 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:47:00.238957 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:47:00.238960 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:47:00.238962 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:47:00.239115 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:47:00.239519 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:47:00.239526 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:47:00.239778 (+ 252us) client_negotiation.cc:769] Sending connection context
0925 02:47:00.239949 (+ 171us) client_negotiation.cc:240] Negotiation successful
0925 02:47:00.240287 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":110,"threads_started":1}
WARNING: no policy specified for kudu/127.13.157.130@KRBTEST.COM; defaulting to no policy
I20260925 02:47:00.242424 32467 heartbeater.cc:347] Connected to a master server at 127.13.157.190:42389
I20260925 02:47:00.242673 32467 heartbeater.cc:464] Registering TS with master...
I20260925 02:47:00.243225 32467 heartbeater.cc:511] Master 127.13.157.190:42389 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.157.130@KRBTEST.COM" created.
I20260925 02:47:00.244800 32276 ts_manager.cc:194] Registered new tserver with Master: 3287457e810f47ecbc499d5a29fcfffd (127.13.157.129:44245)
I20260925 02:47:00.246400 32276 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.157.129@KRBTEST.COM'} at 127.13.157.129:50003
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.157.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.130.keytab.
Entry for principal kudu/127.13.157.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.130.keytab.
I20260925 02:47:00.259629 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790304178325258-13942-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790304178325258-13942-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790304178325258-13942-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790304178325258-13942-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.157.130:0
--local_ip_for_outbound_sockets=127.13.157.130
--webserver_interface=127.13.157.130
--webserver_port=0
--tserver_master_addrs=127.13.157.190:42389
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.130.keytab
--principal=kudu/127.13.157.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:41155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:47:00.368484 32472 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:47:00.368755 32472 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:47:00.368801 32472 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:47:00.372933 32472 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:47:00.373005 32472 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:47:00.373070 32472 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.157.130
I20260925 02:47:00.378484 32472 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:41155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790304178325258-13942-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790304178325258-13942-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.130.keytab
--principal=kudu/127.13.157.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.157.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790304178325258-13942-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.157.130
--webserver_port=0
--tserver_master_addrs=127.13.157.190:42389
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32472
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.157.130
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790304178325258-13942-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:47:00.379642 32472 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:47:00.380685 32472 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:47:00.390534 32477 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:47:00.390583 32478 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:47:00.390583 32480 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:47:00.390765 32472 server_base.cc:1034] running on GCE node
I20260925 02:47:00.391244 32472 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:47:00.391884 32472 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:47:00.393043 32472 hybrid_clock.cc:648] HybridClock initialized: now 1790304420393020 us; error 36 us; skew 500 ppm
Sep 25 02:47:00 dist-test-slave-kznw krb5kdc[32231](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304420, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:47:00.395766 32472 init.cc:377] Logged in from keytab as kudu/127.13.157.130@KRBTEST.COM (short username kudu)
I20260925 02:47:00.396670 32472 webserver.cc:460] Webserver started at http://127.13.157.130:35121/ using document root <none> and password file <none>
I20260925 02:47:00.397213 32472 fs_manager.cc:359] Metadata directory not provided
I20260925 02:47:00.397279 32472 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:47:00.397468 32472 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:47:00.399668 32472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790304178325258-13942-0/minicluster-data/ts-1/data/instance:
uuid: "12e696244e7d4ce2887c099cca595187"
format_stamp: "Formatted at 2026-09-25 02:47:00 on dist-test-slave-kznw"
server_key: "63abcf0b899256ad19103321fdcd935b"
server_key_iv: "e187e369584bf317738de23655016254"
server_key_version: "encryptionkey@0"
I20260925 02:47:00.400228 32472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790304178325258-13942-0/minicluster-data/ts-1/wal/instance:
uuid: "12e696244e7d4ce2887c099cca595187"
format_stamp: "Formatted at 2026-09-25 02:47:00 on dist-test-slave-kznw"
server_key: "63abcf0b899256ad19103321fdcd935b"
server_key_iv: "e187e369584bf317738de23655016254"
server_key_version: "encryptionkey@0"
I20260925 02:47:00.404031 32472 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260925 02:47:00.406469 32487 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:47:00.407826 32472 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260925 02:47:00.407953 32472 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790304178325258-13942-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790304178325258-13942-0/minicluster-data/ts-1/wal
uuid: "12e696244e7d4ce2887c099cca595187"
format_stamp: "Formatted at 2026-09-25 02:47:00 on dist-test-slave-kznw"
server_key: "63abcf0b899256ad19103321fdcd935b"
server_key_iv: "e187e369584bf317738de23655016254"
server_key_version: "encryptionkey@0"
I20260925 02:47:00.408061 32472 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790304178325258-13942-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790304178325258-13942-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790304178325258-13942-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:47:00.425333 32472 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:47:00.428094 32472 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:47:00.428262 32472 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:47:00.428817 32472 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:47:00.429603 32472 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:47:00.429649 32472 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:47:00.429700 32472 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:47:00.429728 32472 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:47:00.438598 32472 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.130:36965
I20260925 02:47:00.438673 32600 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.130:36965 every 8 connection(s)
I20260925 02:47:00.439617 32472 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790304178325258-13942-0/minicluster-data/ts-1/data/info.pb
I20260925 02:47:00.445945 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 32472
I20260925 02:47:00.446069 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790304178325258-13942-0/minicluster-data/ts-1/wal/instance
I20260925 02:47:00.446344 13942 external_mini_cluster.cc:1428] Setting key 4981e521a3b87c87333a190bd7e7b971
Sep 25 02:47:00 dist-test-slave-kznw krb5kdc[32231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304420, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.130@KRBTEST.COM for kudu/127.13.157.190@KRBTEST.COM
I20260925 02:47:00.453891 32315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:00.441566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.130:49433 (local address 127.13.157.190:42389)
0925 02:47:00.441717 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:00.441721 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:00.442551 (+ 830us) server_negotiation.cc:407] Connection header received
0925 02:47:00.444011 (+ 1460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:00.444015 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:00.444078 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:00.444155 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:47:00.445444 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:00.446002 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:00.447334 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:00.447624 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:00.447721 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:00.450272 (+ 2551us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:47:00.450277 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:47:00.450279 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:47:00.450306 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:47:00.451736 (+ 1430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:47:00.452204 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:47:00.452208 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:47:00.452210 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:47:00.452296 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:47:00.452625 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:47:00.452630 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:47:00.452632 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:47:00.452911 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:47:00.453028 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:00.453552 (+ 524us) server_negotiation.cc:299] Negotiation successful
0925 02:47:00.453692 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260925 02:47:00.454559 32603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:00.441895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:42389 (local address 127.13.157.130:49433)
0925 02:47:00.442395 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0925 02:47:00.442440 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0925 02:47:00.443778 (+ 1338us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:47:00.444361 (+ 583us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:47:00.444370 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:47:00.444749 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:47:00.445276 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:47:00.445293 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:47:00.446148 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:47:00.446152 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:47:00.447208 (+ 1056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:47:00.447221 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:47:00.447808 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:47:00.447813 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:47:00.447995 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:00.448618 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:47:00.448633 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:47:00.450105 (+ 1472us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:47:00.451845 (+ 1740us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:47:00.451850 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:47:00.451861 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:47:00.452115 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:47:00.452377 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:47:00.452379 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:47:00.452380 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:47:00.452504 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:47:00.453022 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:47:00.453027 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:47:00.453257 (+ 230us) client_negotiation.cc:769] Sending connection context
0925 02:47:00.453405 (+ 148us) client_negotiation.cc:240] Negotiation successful
0925 02:47:00.453694 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":133,"threads_started":1}
I20260925 02:47:00.455720 32601 heartbeater.cc:347] Connected to a master server at 127.13.157.190:42389
I20260925 02:47:00.455947 32601 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.13.157.131@KRBTEST.COM; defaulting to no policy
I20260925 02:47:00.456432 32601 heartbeater.cc:511] Master 127.13.157.190:42389 requested a full tablet report, sending...
I20260925 02:47:00.457551 32276 ts_manager.cc:194] Registered new tserver with Master: 12e696244e7d4ce2887c099cca595187 (127.13.157.130:36965)
I20260925 02:47:00.458137 32276 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.157.130@KRBTEST.COM'} at 127.13.157.130:49433
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.157.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.157.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.131.keytab.
Entry for principal kudu/127.13.157.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.131.keytab.
I20260925 02:47:00.474186 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790304178325258-13942-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790304178325258-13942-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790304178325258-13942-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790304178325258-13942-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.157.131:0
--local_ip_for_outbound_sockets=127.13.157.131
--webserver_interface=127.13.157.131
--webserver_port=0
--tserver_master_addrs=127.13.157.190:42389
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.131.keytab
--principal=kudu/127.13.157.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:41155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:47:00.586771 32606 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:47:00.587095 32606 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:47:00.587144 32606 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:47:00.591181 32606 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:47:00.591234 32606 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:47:00.591295 32606 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.157.131
I20260925 02:47:00.596580 32606 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:41155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790304178325258-13942-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790304178325258-13942-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.131.keytab
--principal=kudu/127.13.157.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.157.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790304178325258-13942-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.157.131
--webserver_port=0
--tserver_master_addrs=127.13.157.190:42389
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32606
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.157.131
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790304178325258-13942-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:47:00.597605 32606 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:47:00.598563 32606 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:47:00.608208 32612 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:47:00.608245 32606 server_base.cc:1034] running on GCE node
W20260925 02:47:00.608208 32611 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:47:00.608264 32614 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:47:00.608892 32606 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:47:00.609558 32606 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:47:00.610725 32606 hybrid_clock.cc:648] HybridClock initialized: now 1790304420610708 us; error 28 us; skew 500 ppm
Sep 25 02:47:00 dist-test-slave-kznw krb5kdc[32231](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304420, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:47:00.617738 32606 init.cc:377] Logged in from keytab as kudu/127.13.157.131@KRBTEST.COM (short username kudu)
I20260925 02:47:00.618682 32606 webserver.cc:460] Webserver started at http://127.13.157.131:32815/ using document root <none> and password file <none>
I20260925 02:47:00.619309 32606 fs_manager.cc:359] Metadata directory not provided
I20260925 02:47:00.619380 32606 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:47:00.619596 32606 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:47:00.621654 32606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790304178325258-13942-0/minicluster-data/ts-2/data/instance:
uuid: "1f9ea2670262477a83e1f1888b1b7851"
format_stamp: "Formatted at 2026-09-25 02:47:00 on dist-test-slave-kznw"
server_key: "d5472dfac4bfb5936d72bc3ca78b668f"
server_key_iv: "7aca924337a5e0f47965fbc0ce3e35e5"
server_key_version: "encryptionkey@0"
I20260925 02:47:00.622190 32606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790304178325258-13942-0/minicluster-data/ts-2/wal/instance:
uuid: "1f9ea2670262477a83e1f1888b1b7851"
format_stamp: "Formatted at 2026-09-25 02:47:00 on dist-test-slave-kznw"
server_key: "d5472dfac4bfb5936d72bc3ca78b668f"
server_key_iv: "7aca924337a5e0f47965fbc0ce3e35e5"
server_key_version: "encryptionkey@0"
I20260925 02:47:00.626010 32606 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260925 02:47:00.628501 32621 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:47:00.629714 32606 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260925 02:47:00.629820 32606 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790304178325258-13942-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790304178325258-13942-0/minicluster-data/ts-2/wal
uuid: "1f9ea2670262477a83e1f1888b1b7851"
format_stamp: "Formatted at 2026-09-25 02:47:00 on dist-test-slave-kznw"
server_key: "d5472dfac4bfb5936d72bc3ca78b668f"
server_key_iv: "7aca924337a5e0f47965fbc0ce3e35e5"
server_key_version: "encryptionkey@0"
I20260925 02:47:00.629915 32606 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790304178325258-13942-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790304178325258-13942-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790304178325258-13942-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:47:00.647009 32606 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:47:00.649830 32606 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:47:00.649983 32606 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:47:00.650506 32606 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:47:00.651330 32606 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:47:00.651374 32606 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:47:00.651414 32606 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:47:00.651433 32606 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:47:00.659621 32606 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.131:33129
I20260925 02:47:00.659684 32735 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.131:33129 every 8 connection(s)
I20260925 02:47:00.660503 32606 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790304178325258-13942-0/minicluster-data/ts-2/data/info.pb
Sep 25 02:47:00 dist-test-slave-kznw krb5kdc[32231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304420, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.131@KRBTEST.COM for kudu/127.13.157.190@KRBTEST.COM
I20260925 02:47:00.670859 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 32606
I20260925 02:47:00.671003 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790304178325258-13942-0/minicluster-data/ts-2/wal/instance
I20260925 02:47:00.671321 13942 external_mini_cluster.cc:1428] Setting key ff6d07d0ee959fb9475896168da14ca5
I20260925 02:47:00.673583 32315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:00.662311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.131:57335 (local address 127.13.157.190:42389)
0925 02:47:00.662445 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:00.662449 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:00.663289 (+ 840us) server_negotiation.cc:407] Connection header received
0925 02:47:00.664598 (+ 1309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:00.664601 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:00.664660 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:00.664734 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:47:00.665873 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:00.666404 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:00.667543 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:00.667764 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:00.667852 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:00.670181 (+ 2329us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:47:00.670184 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:47:00.670186 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:47:00.670210 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:47:00.671607 (+ 1397us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:47:00.672139 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:47:00.672142 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:47:00.672144 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:47:00.672222 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:47:00.672531 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:47:00.672535 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:47:00.672536 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:47:00.672741 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:47:00.672836 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:00.673329 (+ 493us) server_negotiation.cc:299] Negotiation successful
0925 02:47:00.673457 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260925 02:47:00.674444 32738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:00.662675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:42389 (local address 127.13.157.131:57335)
0925 02:47:00.663140 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0925 02:47:00.663185 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0925 02:47:00.664351 (+ 1166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:47:00.664910 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:47:00.664918 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:47:00.665256 (+ 338us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:47:00.665730 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:47:00.665743 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:47:00.666527 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:47:00.666529 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:47:00.667427 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:47:00.667437 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:47:00.667903 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:47:00.667908 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:47:00.668066 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:00.668612 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:47:00.668628 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:47:00.670018 (+ 1390us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:47:00.671722 (+ 1704us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:47:00.671728 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:47:00.671743 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:47:00.672058 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:47:00.672302 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:47:00.672305 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:47:00.672308 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:47:00.672442 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:47:00.672845 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:47:00.672850 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:47:00.673094 (+ 244us) client_negotiation.cc:769] Sending connection context
0925 02:47:00.673274 (+ 180us) client_negotiation.cc:240] Negotiation successful
0925 02:47:00.673580 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":107,"threads_started":1}
I20260925 02:47:00.675575 32736 heartbeater.cc:347] Connected to a master server at 127.13.157.190:42389
I20260925 02:47:00.675801 32736 heartbeater.cc:464] Registering TS with master...
I20260925 02:47:00.676290 32736 heartbeater.cc:511] Master 127.13.157.190:42389 requested a full tablet report, sending...
I20260925 02:47:00.677251 32276 ts_manager.cc:194] Registered new tserver with Master: 1f9ea2670262477a83e1f1888b1b7851 (127.13.157.131:33129)
I20260925 02:47:00.677806 32276 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.157.131@KRBTEST.COM'} at 127.13.157.131:57335
I20260925 02:47:00.685190 13942 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 02:47:00.694186 32315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:00.687188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46556 (local address 127.13.157.190:42389)
0925 02:47:00.687331 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:00.687336 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:00.687545 (+ 209us) server_negotiation.cc:407] Connection header received
0925 02:47:00.687592 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:00.687594 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:00.687641 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:00.687705 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:47:00.688342 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:00.688887 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:00.689712 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:00.689916 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:00.689989 (+ 73us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:00.691434 (+ 1445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:47:00.691437 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:47:00.691439 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:47:00.691462 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:47:00.692666 (+ 1204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:47:00.693110 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:47:00.693113 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:47:00.693114 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:47:00.693191 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:47:00.693415 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:47:00.693417 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:47:00.693418 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:47:00.693648 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:47:00.693759 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:00.693948 (+ 189us) server_negotiation.cc:299] Negotiation successful
0925 02:47:00.694029 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260925 02:47:00.697023 32276 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46556:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 02:47:00.700299 32276 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 02:47:00.715214 32750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:00.709527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.130:36965 (local address 127.0.0.1:37360)
0925 02:47:00.709979 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0925 02:47:00.709992 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 02:47:00.710084 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:47:00.710790 (+ 706us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:47:00.710793 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:47:00.710808 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:47:00.710895 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:47:00.710900 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:47:00.712739 (+ 1839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:47:00.712743 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:47:00.713713 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:47:00.713721 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:47:00.714787 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:47:00.714790 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:47:00.714878 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:00.714901 (+ 23us) client_negotiation.cc:769] Sending connection context
0925 02:47:00.714986 (+ 85us) client_negotiation.cc:240] Negotiation successful
0925 02:47:00.715063 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":392,"mutex_wait_us":2,"thread_start_us":61,"threads_started":1}
I20260925 02:47:00.715227 32749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:00.709152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54600 (local address 127.13.157.129:44245)
0925 02:47:00.709599 (+ 447us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:00.709667 (+ 68us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:00.709970 (+ 303us) server_negotiation.cc:407] Connection header received
0925 02:47:00.710219 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:00.710225 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:00.710435 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:00.710566 (+ 131us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:47:00.711007 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:00.712119 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:00.713349 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:00.714200 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:00.714323 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:00.714456 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:00.715001 (+ 545us) server_negotiation.cc:299] Negotiation successful
0925 02:47:00.715106 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":332,"thread_start_us":111,"threads_started":1}
I20260925 02:47:00.715509 32748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:00.709096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44268 (local address 127.13.157.131:33129)
0925 02:47:00.709834 (+ 738us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:00.709845 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:00.710185 (+ 340us) server_negotiation.cc:407] Connection header received
0925 02:47:00.710257 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:00.710262 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:00.710434 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:00.710578 (+ 144us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:47:00.710887 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:00.711898 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:00.713460 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:00.714231 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:00.714327 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:00.714456 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:00.715283 (+ 827us) server_negotiation.cc:299] Negotiation successful
0925 02:47:00.715381 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":644,"thread_start_us":152,"threads_started":1}
I20260925 02:47:00.715809 32752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:00.709888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37360 (local address 127.13.157.130:36965)
0925 02:47:00.710278 (+ 390us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:00.710286 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:00.710304 (+ 18us) server_negotiation.cc:407] Connection header received
0925 02:47:00.710363 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:00.710366 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:00.710506 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:00.710641 (+ 135us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:47:00.711036 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:00.712605 (+ 1569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:00.713820 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:00.714468 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:00.714597 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:00.714725 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:00.715590 (+ 865us) server_negotiation.cc:299] Negotiation successful
0925 02:47:00.715699 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":312,"thread_start_us":98,"threads_started":1}
I20260925 02:47:00.715219 32751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:00.708811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.131:33129 (local address 127.0.0.1:44268)
0925 02:47:00.709902 (+ 1091us) negotiation.cc:107] Waiting for socket to connect
0925 02:47:00.709913 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 02:47:00.710042 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:47:00.710606 (+ 564us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:47:00.710612 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:47:00.710634 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:47:00.710770 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:47:00.710778 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:47:00.712038 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:47:00.712041 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:47:00.713333 (+ 1292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:47:00.713344 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:47:00.714521 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:47:00.714525 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:47:00.714738 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:00.714796 (+ 58us) client_negotiation.cc:769] Sending connection context
0925 02:47:00.714901 (+ 105us) client_negotiation.cc:240] Negotiation successful
0925 02:47:00.715053 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1037,"mutex_wait_us":692,"thread_start_us":93,"threads_started":1}
I20260925 02:47:00.715584 32747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:00.708811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.129:44245 (local address 127.0.0.1:54600)
0925 02:47:00.709816 (+ 1005us) negotiation.cc:107] Waiting for socket to connect
0925 02:47:00.709867 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0925 02:47:00.710058 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:47:00.710740 (+ 682us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:47:00.710742 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:47:00.710757 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:47:00.710860 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:47:00.710866 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:47:00.712265 (+ 1399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:47:00.712268 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:47:00.713235 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:47:00.713245 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:47:00.714521 (+ 1276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:47:00.714525 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:47:00.714741 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:00.714804 (+ 63us) client_negotiation.cc:769] Sending connection context
0925 02:47:00.714901 (+ 97us) client_negotiation.cc:240] Negotiation successful
0925 02:47:00.715478 (+ 577us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":840,"mutex_wait_us":90,"thread_start_us":69,"threads_started":1}
I20260925 02:47:00.718564 32401 tablet_service.cc:1467] Processing CreateTablet for tablet b16f17b58b884c409a356ab0b353a7bf (DEFAULT_TABLE table=test-table [id=736a2ce088244854bc1a157cc39dbb63]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:47:00.719601 32401 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b16f17b58b884c409a356ab0b353a7bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:47:00.720645 32669 tablet_service.cc:1467] Processing CreateTablet for tablet b16f17b58b884c409a356ab0b353a7bf (DEFAULT_TABLE table=test-table [id=736a2ce088244854bc1a157cc39dbb63]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:47:00.721489 32669 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b16f17b58b884c409a356ab0b353a7bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:47:00.723042 32535 tablet_service.cc:1467] Processing CreateTablet for tablet b16f17b58b884c409a356ab0b353a7bf (DEFAULT_TABLE table=test-table [id=736a2ce088244854bc1a157cc39dbb63]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:47:00.723990 32535 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b16f17b58b884c409a356ab0b353a7bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:47:00.727425 32754 tablet_bootstrap.cc:492] T b16f17b58b884c409a356ab0b353a7bf P 1f9ea2670262477a83e1f1888b1b7851: Bootstrap starting.
I20260925 02:47:00.728058 32753 tablet_bootstrap.cc:492] T b16f17b58b884c409a356ab0b353a7bf P 3287457e810f47ecbc499d5a29fcfffd: Bootstrap starting.
I20260925 02:47:00.729348 32754 tablet_bootstrap.cc:654] T b16f17b58b884c409a356ab0b353a7bf P 1f9ea2670262477a83e1f1888b1b7851: Neither blocks nor log segments found. Creating new log.
I20260925 02:47:00.730249 32753 tablet_bootstrap.cc:654] T b16f17b58b884c409a356ab0b353a7bf P 3287457e810f47ecbc499d5a29fcfffd: Neither blocks nor log segments found. Creating new log.
I20260925 02:47:00.730386 32754 log.cc:824] T b16f17b58b884c409a356ab0b353a7bf P 1f9ea2670262477a83e1f1888b1b7851: Log is configured to *not* fsync() on all Append() calls
I20260925 02:47:00.730769 32755 tablet_bootstrap.cc:492] T b16f17b58b884c409a356ab0b353a7bf P 12e696244e7d4ce2887c099cca595187: Bootstrap starting.
I20260925 02:47:00.731498 32753 log.cc:824] T b16f17b58b884c409a356ab0b353a7bf P 3287457e810f47ecbc499d5a29fcfffd: Log is configured to *not* fsync() on all Append() calls
I20260925 02:47:00.732585 32754 tablet_bootstrap.cc:492] T b16f17b58b884c409a356ab0b353a7bf P 1f9ea2670262477a83e1f1888b1b7851: No bootstrap required, opened a new log
I20260925 02:47:00.732777 32754 ts_tablet_manager.cc:1397] T b16f17b58b884c409a356ab0b353a7bf P 1f9ea2670262477a83e1f1888b1b7851: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260925 02:47:00.733158 32755 tablet_bootstrap.cc:654] T b16f17b58b884c409a356ab0b353a7bf P 12e696244e7d4ce2887c099cca595187: Neither blocks nor log segments found. Creating new log.
I20260925 02:47:00.733233 32753 tablet_bootstrap.cc:492] T b16f17b58b884c409a356ab0b353a7bf P 3287457e810f47ecbc499d5a29fcfffd: No bootstrap required, opened a new log
I20260925 02:47:00.733413 32753 ts_tablet_manager.cc:1397] T b16f17b58b884c409a356ab0b353a7bf P 3287457e810f47ecbc499d5a29fcfffd: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260925 02:47:00.734396 32755 log.cc:824] T b16f17b58b884c409a356ab0b353a7bf P 12e696244e7d4ce2887c099cca595187: Log is configured to *not* fsync() on all Append() calls
I20260925 02:47:00.736352 32755 tablet_bootstrap.cc:492] T b16f17b58b884c409a356ab0b353a7bf P 12e696244e7d4ce2887c099cca595187: No bootstrap required, opened a new log
I20260925 02:47:00.736533 32755 ts_tablet_manager.cc:1397] T b16f17b58b884c409a356ab0b353a7bf P 12e696244e7d4ce2887c099cca595187: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260925 02:47:00.736357 32754 raft_consensus.cc:348] T b16f17b58b884c409a356ab0b353a7bf P 1f9ea2670262477a83e1f1888b1b7851 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3287457e810f47ecbc499d5a29fcfffd" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 44245 } } peers { permanent_uuid: "1f9ea2670262477a83e1f1888b1b7851" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 33129 } } peers { permanent_uuid: "12e696244e7d4ce2887c099cca595187" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 36965 } }
I20260925 02:47:00.736624 32754 raft_consensus.cc:374] T b16f17b58b884c409a356ab0b353a7bf P 1f9ea2670262477a83e1f1888b1b7851 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:47:00.736675 32754 raft_consensus.cc:729] T b16f17b58b884c409a356ab0b353a7bf P 1f9ea2670262477a83e1f1888b1b7851 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f9ea2670262477a83e1f1888b1b7851, State: Initialized, Role: FOLLOWER
I20260925 02:47:00.736886 32753 raft_consensus.cc:348] T b16f17b58b884c409a356ab0b353a7bf P 3287457e810f47ecbc499d5a29fcfffd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3287457e810f47ecbc499d5a29fcfffd" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 44245 } } peers { permanent_uuid: "1f9ea2670262477a83e1f1888b1b7851" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 33129 } } peers { permanent_uuid: "12e696244e7d4ce2887c099cca595187" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 36965 } }
I20260925 02:47:00.737133 32753 raft_consensus.cc:374] T b16f17b58b884c409a356ab0b353a7bf P 3287457e810f47ecbc499d5a29fcfffd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:47:00.737181 32753 raft_consensus.cc:729] T b16f17b58b884c409a356ab0b353a7bf P 3287457e810f47ecbc499d5a29fcfffd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3287457e810f47ecbc499d5a29fcfffd, State: Initialized, Role: FOLLOWER
I20260925 02:47:00.737121 32754 consensus_queue.cc:260] T b16f17b58b884c409a356ab0b353a7bf P 1f9ea2670262477a83e1f1888b1b7851 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3287457e810f47ecbc499d5a29fcfffd" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 44245 } } peers { permanent_uuid: "1f9ea2670262477a83e1f1888b1b7851" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 33129 } } peers { permanent_uuid: "12e696244e7d4ce2887c099cca595187" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 36965 } }
I20260925 02:47:00.737612 32753 consensus_queue.cc:260] T b16f17b58b884c409a356ab0b353a7bf P 3287457e810f47ecbc499d5a29fcfffd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3287457e810f47ecbc499d5a29fcfffd" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 44245 } } peers { permanent_uuid: "1f9ea2670262477a83e1f1888b1b7851" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 33129 } } peers { permanent_uuid: "12e696244e7d4ce2887c099cca595187" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 36965 } }
I20260925 02:47:00.738073 32736 heartbeater.cc:503] Master 127.13.157.190:42389 was elected leader, sending a full tablet report...
I20260925 02:47:00.738131 32754 ts_tablet_manager.cc:1428] T b16f17b58b884c409a356ab0b353a7bf P 1f9ea2670262477a83e1f1888b1b7851: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260925 02:47:00.738539 32753 ts_tablet_manager.cc:1428] T b16f17b58b884c409a356ab0b353a7bf P 3287457e810f47ecbc499d5a29fcfffd: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260925 02:47:00.738672 32467 heartbeater.cc:503] Master 127.13.157.190:42389 was elected leader, sending a full tablet report...
I20260925 02:47:00.740059 32755 raft_consensus.cc:348] T b16f17b58b884c409a356ab0b353a7bf P 12e696244e7d4ce2887c099cca595187 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3287457e810f47ecbc499d5a29fcfffd" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 44245 } } peers { permanent_uuid: "1f9ea2670262477a83e1f1888b1b7851" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 33129 } } peers { permanent_uuid: "12e696244e7d4ce2887c099cca595187" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 36965 } }
I20260925 02:47:00.740285 32755 raft_consensus.cc:374] T b16f17b58b884c409a356ab0b353a7bf P 12e696244e7d4ce2887c099cca595187 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:47:00.740336 32755 raft_consensus.cc:729] T b16f17b58b884c409a356ab0b353a7bf P 12e696244e7d4ce2887c099cca595187 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12e696244e7d4ce2887c099cca595187, State: Initialized, Role: FOLLOWER
I20260925 02:47:00.740706 32755 consensus_queue.cc:260] T b16f17b58b884c409a356ab0b353a7bf P 12e696244e7d4ce2887c099cca595187 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3287457e810f47ecbc499d5a29fcfffd" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 44245 } } peers { permanent_uuid: "1f9ea2670262477a83e1f1888b1b7851" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 33129 } } peers { permanent_uuid: "12e696244e7d4ce2887c099cca595187" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 36965 } }
I20260925 02:47:00.741441 32601 heartbeater.cc:503] Master 127.13.157.190:42389 was elected leader, sending a full tablet report...
I20260925 02:47:00.741608 32755 ts_tablet_manager.cc:1428] T b16f17b58b884c409a356ab0b353a7bf P 12e696244e7d4ce2887c099cca595187: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260925 02:47:00.829257 32760 raft_consensus.cc:482] T b16f17b58b884c409a356ab0b353a7bf P 3287457e810f47ecbc499d5a29fcfffd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 02:47:00.829481 32760 raft_consensus.cc:504] T b16f17b58b884c409a356ab0b353a7bf P 3287457e810f47ecbc499d5a29fcfffd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3287457e810f47ecbc499d5a29fcfffd" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 44245 } } peers { permanent_uuid: "1f9ea2670262477a83e1f1888b1b7851" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 33129 } } peers { permanent_uuid: "12e696244e7d4ce2887c099cca595187" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 36965 } }
I20260925 02:47:00.830638 32760 leader_election.cc:288] T b16f17b58b884c409a356ab0b353a7bf P 3287457e810f47ecbc499d5a29fcfffd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f9ea2670262477a83e1f1888b1b7851 (127.13.157.131:33129), 12e696244e7d4ce2887c099cca595187 (127.13.157.130:36965)
I20260925 02:47:00.835152 32752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:00.831091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.129:49015 (local address 127.13.157.130:36965)
0925 02:47:00.831229 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:00.831233 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:00.831435 (+ 202us) server_negotiation.cc:407] Connection header received
0925 02:47:00.831586 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:00.831589 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:00.831641 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:00.831713 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:47:00.832073 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:00.832816 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:00.834119 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:00.834590 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:00.834687 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:00.834760 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:00.834943 (+ 183us) server_negotiation.cc:299] Negotiation successful
0925 02:47:00.835022 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260925 02:47:00.835177 32763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:00.830948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.130:36965 (local address 127.13.157.129:49015)
0925 02:47:00.831347 (+ 399us) negotiation.cc:107] Waiting for socket to connect
0925 02:47:00.831357 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 02:47:00.831469 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:47:00.831729 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:47:00.831732 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:47:00.831746 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:47:00.831936 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:47:00.831943 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:47:00.833010 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:47:00.833014 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:47:00.833994 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:47:00.834004 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:47:00.834709 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:47:00.834713 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:47:00.834809 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:00.834834 (+ 25us) client_negotiation.cc:769] Sending connection context
0925 02:47:00.834922 (+ 88us) client_negotiation.cc:240] Negotiation successful
0925 02:47:00.834994 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":341,"mutex_wait_us":75,"thread_start_us":119,"threads_started":1}
I20260925 02:47:00.835500 32748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:00.831082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.129:34209 (local address 127.13.157.131:33129)
0925 02:47:00.831257 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:00.831262 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:00.831338 (+ 76us) server_negotiation.cc:407] Connection header received
0925 02:47:00.831487 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:00.831490 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:00.831576 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:00.831653 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:47:00.832035 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:00.832882 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:00.834219 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:00.834644 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:00.834757 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:00.834807 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:00.835293 (+ 486us) server_negotiation.cc:299] Negotiation successful
0925 02:47:00.835373 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260925 02:47:00.835213 32762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:00.830923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.131:33129 (local address 127.13.157.129:34209)
0925 02:47:00.831255 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0925 02:47:00.831270 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 02:47:00.831389 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:47:00.831665 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:47:00.831668 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:47:00.831683 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:47:00.831895 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:47:00.831903 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:47:00.833007 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:47:00.833011 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:47:00.834098 (+ 1087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:47:00.834109 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:47:00.834763 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:47:00.834768 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:47:00.834861 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:00.834882 (+ 21us) client_negotiation.cc:769] Sending connection context
0925 02:47:00.834981 (+ 99us) client_negotiation.cc:240] Negotiation successful
0925 02:47:00.835053 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"mutex_wait_us":107,"thread_start_us":105,"threads_started":1}
I20260925 02:47:00.836146 32555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b16f17b58b884c409a356ab0b353a7bf" candidate_uuid: "3287457e810f47ecbc499d5a29fcfffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12e696244e7d4ce2887c099cca595187" is_pre_election: true
I20260925 02:47:00.836475 32555 raft_consensus.cc:2454] T b16f17b58b884c409a356ab0b353a7bf P 12e696244e7d4ce2887c099cca595187 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3287457e810f47ecbc499d5a29fcfffd in term 0.
I20260925 02:47:00.836956 32354 leader_election.cc:302] T b16f17b58b884c409a356ab0b353a7bf P 3287457e810f47ecbc499d5a29fcfffd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12e696244e7d4ce2887c099cca595187, 3287457e810f47ecbc499d5a29fcfffd; no voters:
I20260925 02:47:00.837091 32689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b16f17b58b884c409a356ab0b353a7bf" candidate_uuid: "3287457e810f47ecbc499d5a29fcfffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f9ea2670262477a83e1f1888b1b7851" is_pre_election: true
I20260925 02:47:00.837224 32760 raft_consensus.cc:2781] T b16f17b58b884c409a356ab0b353a7bf P 3287457e810f47ecbc499d5a29fcfffd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 02:47:00.837317 32760 raft_consensus.cc:482] T b16f17b58b884c409a356ab0b353a7bf P 3287457e810f47ecbc499d5a29fcfffd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 02:47:00.837364 32760 raft_consensus.cc:3037] T b16f17b58b884c409a356ab0b353a7bf P 3287457e810f47ecbc499d5a29fcfffd [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:47:00.837386 32689 raft_consensus.cc:2454] T b16f17b58b884c409a356ab0b353a7bf P 1f9ea2670262477a83e1f1888b1b7851 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3287457e810f47ecbc499d5a29fcfffd in term 0.
I20260925 02:47:00.838546 32760 raft_consensus.cc:504] T b16f17b58b884c409a356ab0b353a7bf P 3287457e810f47ecbc499d5a29fcfffd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3287457e810f47ecbc499d5a29fcfffd" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 44245 } } peers { permanent_uuid: "1f9ea2670262477a83e1f1888b1b7851" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 33129 } } peers { permanent_uuid: "12e696244e7d4ce2887c099cca595187" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 36965 } }
I20260925 02:47:00.838864 32760 leader_election.cc:288] T b16f17b58b884c409a356ab0b353a7bf P 3287457e810f47ecbc499d5a29fcfffd [CANDIDATE]: Term 1 election: Requested vote from peers 1f9ea2670262477a83e1f1888b1b7851 (127.13.157.131:33129), 12e696244e7d4ce2887c099cca595187 (127.13.157.130:36965)
I20260925 02:47:00.839226 32689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b16f17b58b884c409a356ab0b353a7bf" candidate_uuid: "3287457e810f47ecbc499d5a29fcfffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f9ea2670262477a83e1f1888b1b7851"
I20260925 02:47:00.839247 32555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b16f17b58b884c409a356ab0b353a7bf" candidate_uuid: "3287457e810f47ecbc499d5a29fcfffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12e696244e7d4ce2887c099cca595187"
I20260925 02:47:00.839334 32689 raft_consensus.cc:3037] T b16f17b58b884c409a356ab0b353a7bf P 1f9ea2670262477a83e1f1888b1b7851 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:47:00.839372 32555 raft_consensus.cc:3037] T b16f17b58b884c409a356ab0b353a7bf P 12e696244e7d4ce2887c099cca595187 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:47:00.840401 32555 raft_consensus.cc:2454] T b16f17b58b884c409a356ab0b353a7bf P 12e696244e7d4ce2887c099cca595187 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3287457e810f47ecbc499d5a29fcfffd in term 1.
I20260925 02:47:00.840464 32689 raft_consensus.cc:2454] T b16f17b58b884c409a356ab0b353a7bf P 1f9ea2670262477a83e1f1888b1b7851 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3287457e810f47ecbc499d5a29fcfffd in term 1.
I20260925 02:47:00.840665 32354 leader_election.cc:302] T b16f17b58b884c409a356ab0b353a7bf P 3287457e810f47ecbc499d5a29fcfffd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12e696244e7d4ce2887c099cca595187, 3287457e810f47ecbc499d5a29fcfffd; no voters:
I20260925 02:47:00.840839 32760 raft_consensus.cc:2781] T b16f17b58b884c409a356ab0b353a7bf P 3287457e810f47ecbc499d5a29fcfffd [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:47:00.841079 32760 raft_consensus.cc:686] T b16f17b58b884c409a356ab0b353a7bf P 3287457e810f47ecbc499d5a29fcfffd [term 1 LEADER]: Becoming Leader. State: Replica: 3287457e810f47ecbc499d5a29fcfffd, State: Running, Role: LEADER
I20260925 02:47:00.841423 32760 consensus_queue.cc:237] T b16f17b58b884c409a356ab0b353a7bf P 3287457e810f47ecbc499d5a29fcfffd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3287457e810f47ecbc499d5a29fcfffd" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 44245 } } peers { permanent_uuid: "1f9ea2670262477a83e1f1888b1b7851" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 33129 } } peers { permanent_uuid: "12e696244e7d4ce2887c099cca595187" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 36965 } }
I20260925 02:47:00.844645 32276 catalog_manager.cc:5495] T b16f17b58b884c409a356ab0b353a7bf P 3287457e810f47ecbc499d5a29fcfffd reported cstate change: term changed from 0 to 1, leader changed from <none> to 3287457e810f47ecbc499d5a29fcfffd (127.13.157.129). New cstate: current_term: 1 leader_uuid: "3287457e810f47ecbc499d5a29fcfffd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3287457e810f47ecbc499d5a29fcfffd" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 44245 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1f9ea2670262477a83e1f1888b1b7851" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 33129 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12e696244e7d4ce2887c099cca595187" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 36965 } health_report { overall_health: UNKNOWN } } }
I20260925 02:47:00.874123 32749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:00.870483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54608 (local address 127.13.157.129:44245)
0925 02:47:00.870623 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:00.870628 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:00.870670 (+ 42us) server_negotiation.cc:407] Connection header received
0925 02:47:00.870849 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:00.870852 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:00.870934 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:00.871050 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 02:47:00.871312 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:00.871824 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:00.872688 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:00.872899 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:00.872997 (+ 98us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:00.873294 (+ 297us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 02:47:00.873706 (+ 412us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 02:47:00.873837 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:00.873941 (+ 104us) server_negotiation.cc:299] Negotiation successful
0925 02:47:00.873990 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260925 02:47:00.879908 32555 raft_consensus.cc:1261] T b16f17b58b884c409a356ab0b353a7bf P 12e696244e7d4ce2887c099cca595187 [term 1 FOLLOWER]: Refusing update from remote peer 3287457e810f47ecbc499d5a29fcfffd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:47:00.879925 32689 raft_consensus.cc:1261] T b16f17b58b884c409a356ab0b353a7bf P 1f9ea2670262477a83e1f1888b1b7851 [term 1 FOLLOWER]: Refusing update from remote peer 3287457e810f47ecbc499d5a29fcfffd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:47:00.880610 32764 consensus_queue.cc:1035] T b16f17b58b884c409a356ab0b353a7bf P 3287457e810f47ecbc499d5a29fcfffd [LEADER]: Connected to new peer: Peer: permanent_uuid: "12e696244e7d4ce2887c099cca595187" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 36965 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:47:00.880800 32760 consensus_queue.cc:1035] T b16f17b58b884c409a356ab0b353a7bf P 3287457e810f47ecbc499d5a29fcfffd [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f9ea2670262477a83e1f1888b1b7851" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 33129 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:47:00.896840 32275 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46556:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 02:47:00.897017 32275 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46556:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 02:47:00.899380 32275 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c65e2f3f88d149928e6946de48948cfd: Sending DeleteTablet for 3 replicas of tablet b16f17b58b884c409a356ab0b353a7bf
I20260925 02:47:00.900338 32401 tablet_service.cc:1514] Processing DeleteTablet for tablet b16f17b58b884c409a356ab0b353a7bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:47:00 UTC) from {username='kudu', principal='kudu/127.13.157.190@KRBTEST.COM'} at 127.0.0.1:54600
I20260925 02:47:00.900851 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 32338
I20260925 02:47:00.900913 32535 tablet_service.cc:1514] Processing DeleteTablet for tablet b16f17b58b884c409a356ab0b353a7bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:47:00 UTC) from {username='kudu', principal='kudu/127.13.157.190@KRBTEST.COM'} at 127.0.0.1:37360
I20260925 02:47:00.902278 32669 tablet_service.cc:1514] Processing DeleteTablet for tablet b16f17b58b884c409a356ab0b353a7bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:47:00 UTC) from {username='kudu', principal='kudu/127.13.157.190@KRBTEST.COM'} at 127.0.0.1:44268
I20260925 02:47:00.902704 311 tablet_replica.cc:330] T b16f17b58b884c409a356ab0b353a7bf P 12e696244e7d4ce2887c099cca595187: stopping tablet replica
I20260925 02:47:00.902733 312 tablet_replica.cc:330] T b16f17b58b884c409a356ab0b353a7bf P 1f9ea2670262477a83e1f1888b1b7851: stopping tablet replica
I20260925 02:47:00.903087 311 raft_consensus.cc:2229] T b16f17b58b884c409a356ab0b353a7bf P 12e696244e7d4ce2887c099cca595187 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 02:47:00.903087 312 raft_consensus.cc:2229] T b16f17b58b884c409a356ab0b353a7bf P 1f9ea2670262477a83e1f1888b1b7851 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 02:47:00.903430 311 raft_consensus.cc:2258] T b16f17b58b884c409a356ab0b353a7bf P 12e696244e7d4ce2887c099cca595187 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:47:00.903432 312 raft_consensus.cc:2258] T b16f17b58b884c409a356ab0b353a7bf P 1f9ea2670262477a83e1f1888b1b7851 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:47:00.905056 311 ts_tablet_manager.cc:1905] T b16f17b58b884c409a356ab0b353a7bf P 12e696244e7d4ce2887c099cca595187: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:47:00.905057 312 ts_tablet_manager.cc:1905] T b16f17b58b884c409a356ab0b353a7bf P 1f9ea2670262477a83e1f1888b1b7851: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:47:00.907732 312 ts_tablet_manager.cc:1918] T b16f17b58b884c409a356ab0b353a7bf P 1f9ea2670262477a83e1f1888b1b7851: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 02:47:00.907821 312 log.cc:1196] T b16f17b58b884c409a356ab0b353a7bf P 1f9ea2670262477a83e1f1888b1b7851: Deleting WAL directory at /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790304178325258-13942-0/minicluster-data/ts-2/wal/wals/b16f17b58b884c409a356ab0b353a7bf
W20260925 02:47:00.907869 32260 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.157.129:44245 (error 108)
I20260925 02:47:00.908030 311 ts_tablet_manager.cc:1918] T b16f17b58b884c409a356ab0b353a7bf P 12e696244e7d4ce2887c099cca595187: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 02:47:00.908125 311 log.cc:1196] T b16f17b58b884c409a356ab0b353a7bf P 12e696244e7d4ce2887c099cca595187: Deleting WAL directory at /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790304178325258-13942-0/minicluster-data/ts-1/wal/wals/b16f17b58b884c409a356ab0b353a7bf
I20260925 02:47:00.908138 312 ts_tablet_manager.cc:1939] T b16f17b58b884c409a356ab0b353a7bf P 1f9ea2670262477a83e1f1888b1b7851: Deleting consensus metadata
I20260925 02:47:00.908391 311 ts_tablet_manager.cc:1939] T b16f17b58b884c409a356ab0b353a7bf P 12e696244e7d4ce2887c099cca595187: Deleting consensus metadata
I20260925 02:47:00.908681 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 32472
I20260925 02:47:00.908981 32751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:00.908710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.129:44245 (local address 127.0.0.1:54622)
0925 02:47:00.908828 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0925 02:47:00.908891 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.129:44245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260925 02:47:00.910152 32261 catalog_manager.cc:4841] TS 1f9ea2670262477a83e1f1888b1b7851 (127.13.157.131:33129): tablet b16f17b58b884c409a356ab0b353a7bf (table test-table [id=736a2ce088244854bc1a157cc39dbb63]) successfully deleted
W20260925 02:47:00.910537 32260 catalog_manager.cc:4584] TS 3287457e810f47ecbc499d5a29fcfffd (127.13.157.129:44245): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b16f17b58b884c409a356ab0b353a7bf: Network error: Client connection negotiation failed: client connection to 127.13.157.129:44245: connect: Connection refused (error 111)
I20260925 02:47:00.913923 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 32606
I20260925 02:47:00.914412 32751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:00.914135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.130:36965 (local address 127.0.0.1:37364)
0925 02:47:00.914280 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0925 02:47:00.914326 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.130:36965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260925 02:47:00.918893 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 32244
2026-09-25T02:47:00Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2214 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-09-25T02:47:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T02:47:00Z Disabled control of system clock
I20260925 02:47:00.948344 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790304178325258-13942-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790304178325258-13942-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790304178325258-13942-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.157.190:44603
--webserver_interface=127.13.157.190
--webserver_port=0
--builtin_ntp_servers=127.13.157.148:39171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.157.190:44603
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260925 02:47:01.058956 317 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:47:01.059250 317 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:47:01.059286 317 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:47:01.063371 317 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:47:01.063431 317 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:47:01.063452 317 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:47:01.063467 317 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:47:01.063481 317 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:47:01.068495 317 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:39171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790304178325258-13942-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790304178325258-13942-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.157.190:44603
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.157.190:44603
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.157.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.317
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790304178325258-13942-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:47:01.069621 317 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:47:01.071051 317 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:47:01.079596 322 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:47:01.079605 325 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:47:01.079627 323 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:47:01.079873 317 server_base.cc:1034] running on GCE node
I20260925 02:47:01.080643 317 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:47:01.081835 317 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:47:01.082993 317 hybrid_clock.cc:648] HybridClock initialized: now 1790304421082974 us; error 31 us; skew 500 ppm
I20260925 02:47:01.084932 317 webserver.cc:460] Webserver started at http://127.13.157.190:45711/ using document root <none> and password file <none>
I20260925 02:47:01.085492 317 fs_manager.cc:359] Metadata directory not provided
I20260925 02:47:01.085573 317 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:47:01.085783 317 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:47:01.087895 317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790304178325258-13942-0/minicluster-data/master-0/data/instance:
uuid: "7499a16eaeaf4ab797b137a223d937d7"
format_stamp: "Formatted at 2026-09-25 02:47:01 on dist-test-slave-kznw"
server_key: "41ee8386e5b1d4159a37c613873ee5b9"
server_key_iv: "4358e13309342f25d1a0cb4b44b8998a"
server_key_version: "encryptionkey@0"
I20260925 02:47:01.088438 317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790304178325258-13942-0/minicluster-data/master-0/wal/instance:
uuid: "7499a16eaeaf4ab797b137a223d937d7"
format_stamp: "Formatted at 2026-09-25 02:47:01 on dist-test-slave-kznw"
server_key: "41ee8386e5b1d4159a37c613873ee5b9"
server_key_iv: "4358e13309342f25d1a0cb4b44b8998a"
server_key_version: "encryptionkey@0"
I20260925 02:47:01.092105 317 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.004s
I20260925 02:47:01.094718 331 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:47:01.096035 317 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260925 02:47:01.096151 317 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790304178325258-13942-0/minicluster-data/master-0/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790304178325258-13942-0/minicluster-data/master-0/wal
uuid: "7499a16eaeaf4ab797b137a223d937d7"
format_stamp: "Formatted at 2026-09-25 02:47:01 on dist-test-slave-kznw"
server_key: "41ee8386e5b1d4159a37c613873ee5b9"
server_key_iv: "4358e13309342f25d1a0cb4b44b8998a"
server_key_version: "encryptionkey@0"
I20260925 02:47:01.096249 317 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790304178325258-13942-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790304178325258-13942-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790304178325258-13942-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:47:01.112879 317 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:47:01.113595 317 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:47:01.113776 317 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:47:01.120649 317 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.190:44603
I20260925 02:47:01.120721 384 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.190:44603 every 8 connection(s)
I20260925 02:47:01.121625 317 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
I20260925 02:47:01.124159 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 317
I20260925 02:47:01.124274 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790304178325258-13942-0/minicluster-data/master-0/wal/instance
I20260925 02:47:01.124593 13942 external_mini_cluster.cc:1428] Setting key 6bc4a9accf9bfe3fb01dec39ad14cf93
I20260925 02:47:01.124953 385 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:47:01.132380 385 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7499a16eaeaf4ab797b137a223d937d7: Bootstrap starting.
I20260925 02:47:01.135175 385 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7499a16eaeaf4ab797b137a223d937d7: Neither blocks nor log segments found. Creating new log.
I20260925 02:47:01.135645 388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:01.126119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51328 (local address 127.13.157.190:44603)
0925 02:47:01.126835 (+ 716us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:01.126855 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:01.126895 (+ 40us) server_negotiation.cc:407] Connection header received
0925 02:47:01.127830 (+ 935us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:01.127855 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:01.128395 (+ 540us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:01.128767 (+ 372us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:47:01.129232 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:01.130713 (+ 1481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:01.131694 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:01.132381 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:01.132556 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:01.133450 (+ 894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:47:01.133456 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:47:01.133471 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:47:01.134012 (+ 541us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:47:01.134079 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:47:01.134105 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:47:01.134197 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:01.134337 (+ 140us) server_negotiation.cc:299] Negotiation successful
0925 02:47:01.134617 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":133,"threads_started":1}
I20260925 02:47:01.136092 385 log.cc:824] T 00000000000000000000000000000000 P 7499a16eaeaf4ab797b137a223d937d7: Log is configured to *not* fsync() on all Append() calls
I20260925 02:47:01.139076 385 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7499a16eaeaf4ab797b137a223d937d7: No bootstrap required, opened a new log
I20260925 02:47:01.142855 385 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7499a16eaeaf4ab797b137a223d937d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7499a16eaeaf4ab797b137a223d937d7" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 44603 } }
I20260925 02:47:01.143151 385 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7499a16eaeaf4ab797b137a223d937d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:47:01.143209 385 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7499a16eaeaf4ab797b137a223d937d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7499a16eaeaf4ab797b137a223d937d7, State: Initialized, Role: FOLLOWER
I20260925 02:47:01.143638 385 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7499a16eaeaf4ab797b137a223d937d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7499a16eaeaf4ab797b137a223d937d7" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 44603 } }
I20260925 02:47:01.143762 385 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7499a16eaeaf4ab797b137a223d937d7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 02:47:01.143822 385 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7499a16eaeaf4ab797b137a223d937d7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 02:47:01.143919 385 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7499a16eaeaf4ab797b137a223d937d7 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:47:01.144913 385 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7499a16eaeaf4ab797b137a223d937d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7499a16eaeaf4ab797b137a223d937d7" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 44603 } }
I20260925 02:47:01.145264 385 leader_election.cc:302] T 00000000000000000000000000000000 P 7499a16eaeaf4ab797b137a223d937d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7499a16eaeaf4ab797b137a223d937d7; no voters:
I20260925 02:47:01.145571 385 leader_election.cc:288] T 00000000000000000000000000000000 P 7499a16eaeaf4ab797b137a223d937d7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 02:47:01.145668 390 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7499a16eaeaf4ab797b137a223d937d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:47:01.145843 390 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7499a16eaeaf4ab797b137a223d937d7 [term 1 LEADER]: Becoming Leader. State: Replica: 7499a16eaeaf4ab797b137a223d937d7, State: Running, Role: LEADER
I20260925 02:47:01.146142 390 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7499a16eaeaf4ab797b137a223d937d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7499a16eaeaf4ab797b137a223d937d7" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 44603 } }
I20260925 02:47:01.146946 385 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7499a16eaeaf4ab797b137a223d937d7 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:47:01.147917 392 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7499a16eaeaf4ab797b137a223d937d7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7499a16eaeaf4ab797b137a223d937d7. Latest consensus state: current_term: 1 leader_uuid: "7499a16eaeaf4ab797b137a223d937d7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7499a16eaeaf4ab797b137a223d937d7" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 44603 } } }
I20260925 02:47:01.148046 392 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7499a16eaeaf4ab797b137a223d937d7 [sys.catalog]: This master's current role is: LEADER
I20260925 02:47:01.148325 391 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7499a16eaeaf4ab797b137a223d937d7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7499a16eaeaf4ab797b137a223d937d7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7499a16eaeaf4ab797b137a223d937d7" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 44603 } } }
I20260925 02:47:01.148418 391 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7499a16eaeaf4ab797b137a223d937d7 [sys.catalog]: This master's current role is: LEADER
I20260925 02:47:01.148530 399 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:47:01.151477 399 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:47:01.157186 399 catalog_manager.cc:1347] Generated new cluster ID: b08626c9a682483ea67285a1466db759
I20260925 02:47:01.157249 399 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 02:47:01.181469 399 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 02:47:01.182420 399 catalog_manager.cc:1504] Loading token signing keys...
I20260925 02:47:01.189112 399 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7499a16eaeaf4ab797b137a223d937d7: Generated new TSK 0
I20260925 02:47:01.189772 399 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 02:47:01.199147 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790304178325258-13942-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790304178325258-13942-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790304178325258-13942-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790304178325258-13942-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.157.129:0
--local_ip_for_outbound_sockets=127.13.157.129
--webserver_interface=127.13.157.129
--webserver_port=0
--tserver_master_addrs=127.13.157.190:44603
--builtin_ntp_servers=127.13.157.148:39171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260925 02:47:01.313841 409 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:47:01.314136 409 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:47:01.314178 409 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:47:01.318463 409 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:47:01.318539 409 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:47:01.318595 409 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.157.129
I20260925 02:47:01.323580 409 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:39171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790304178325258-13942-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790304178325258-13942-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.157.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790304178325258-13942-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.157.129
--webserver_port=0
--tserver_master_addrs=127.13.157.190:44603
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.409
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.157.129
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790304178325258-13942-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:47:01.324693 409 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:47:01.326110 409 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:47:01.335969 415 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:47:01.335969 417 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:47:01.335966 414 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:47:01.336230 409 server_base.cc:1034] running on GCE node
I20260925 02:47:01.336783 409 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:47:01.337484 409 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:47:01.338683 409 hybrid_clock.cc:648] HybridClock initialized: now 1790304421338651 us; error 44 us; skew 500 ppm
I20260925 02:47:01.340790 409 webserver.cc:460] Webserver started at http://127.13.157.129:43801/ using document root <none> and password file <none>
I20260925 02:47:01.341362 409 fs_manager.cc:359] Metadata directory not provided
I20260925 02:47:01.341434 409 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:47:01.341627 409 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:47:01.343794 409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790304178325258-13942-0/minicluster-data/ts-0/data/instance:
uuid: "7a8dbdf4dc9547abb9bcd3f62ffdaf54"
format_stamp: "Formatted at 2026-09-25 02:47:01 on dist-test-slave-kznw"
server_key: "ac451417e3a28f85f88adafb4ed6801f"
server_key_iv: "23ebe9ca79da166c09165a8aefe994d0"
server_key_version: "encryptionkey@0"
I20260925 02:47:01.344339 409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790304178325258-13942-0/minicluster-data/ts-0/wal/instance:
uuid: "7a8dbdf4dc9547abb9bcd3f62ffdaf54"
format_stamp: "Formatted at 2026-09-25 02:47:01 on dist-test-slave-kznw"
server_key: "ac451417e3a28f85f88adafb4ed6801f"
server_key_iv: "23ebe9ca79da166c09165a8aefe994d0"
server_key_version: "encryptionkey@0"
I20260925 02:47:01.348133 409 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260925 02:47:01.350708 423 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:47:01.352087 409 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260925 02:47:01.352206 409 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790304178325258-13942-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790304178325258-13942-0/minicluster-data/ts-0/wal
uuid: "7a8dbdf4dc9547abb9bcd3f62ffdaf54"
format_stamp: "Formatted at 2026-09-25 02:47:01 on dist-test-slave-kznw"
server_key: "ac451417e3a28f85f88adafb4ed6801f"
server_key_iv: "23ebe9ca79da166c09165a8aefe994d0"
server_key_version: "encryptionkey@0"
I20260925 02:47:01.352325 409 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790304178325258-13942-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790304178325258-13942-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790304178325258-13942-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:47:01.372304 409 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:47:01.373080 409 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:47:01.373272 409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:47:01.373945 409 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:47:01.375043 409 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:47:01.375097 409 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:47:01.375138 409 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:47:01.375167 409 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:47:01.383940 409 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.129:44363
I20260925 02:47:01.384034 536 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.129:44363 every 8 connection(s)
I20260925 02:47:01.384923 409 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790304178325258-13942-0/minicluster-data/ts-0/data/info.pb
I20260925 02:47:01.385881 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 409
I20260925 02:47:01.385993 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790304178325258-13942-0/minicluster-data/ts-0/wal/instance
I20260925 02:47:01.386270 13942 external_mini_cluster.cc:1428] Setting key 866f3e3dc988a5afd2a0f0d164fcaa35
I20260925 02:47:01.388588 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790304178325258-13942-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790304178325258-13942-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790304178325258-13942-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790304178325258-13942-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.157.130:0
--local_ip_for_outbound_sockets=127.13.157.130
--webserver_interface=127.13.157.130
--webserver_port=0
--tserver_master_addrs=127.13.157.190:44603
--builtin_ntp_servers=127.13.157.148:39171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260925 02:47:01.395176 388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:01.387540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.129:44887 (local address 127.13.157.190:44603)
0925 02:47:01.387664 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:01.387668 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:01.389009 (+ 1341us) server_negotiation.cc:407] Connection header received
0925 02:47:01.390449 (+ 1440us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:01.390452 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:01.390509 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:01.390581 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:47:01.391548 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:01.392112 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:01.393274 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:01.393505 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:01.393595 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:01.394672 (+ 1077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:47:01.394676 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:47:01.394679 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:47:01.394694 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:47:01.394714 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:47:01.394722 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:47:01.394780 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:01.394948 (+ 168us) server_negotiation.cc:299] Negotiation successful
0925 02:47:01.395023 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260925 02:47:01.396095 539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:01.388038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:44603 (local address 127.13.157.129:44887)
0925 02:47:01.388848 (+ 810us) negotiation.cc:107] Waiting for socket to connect
0925 02:47:01.388901 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0925 02:47:01.390226 (+ 1325us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:47:01.390767 (+ 541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:47:01.390774 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:47:01.390865 (+ 91us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:47:01.391407 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:47:01.391421 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:47:01.392247 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:47:01.392250 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:47:01.393156 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:47:01.393166 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:47:01.393650 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:47:01.393654 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:47:01.393819 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:01.394403 (+ 584us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 02:47:01.394405 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:47:01.394450 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 02:47:01.394518 (+ 68us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:47:01.394791 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:47:01.394801 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:47:01.394807 (+ 6us) client_negotiation.cc:769] Sending connection context
0925 02:47:01.394948 (+ 141us) client_negotiation.cc:240] Negotiation successful
0925 02:47:01.395252 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":156,"threads_started":1}
I20260925 02:47:01.397385 537 heartbeater.cc:347] Connected to a master server at 127.13.157.190:44603
I20260925 02:47:01.397688 537 heartbeater.cc:464] Registering TS with master...
I20260925 02:47:01.398217 537 heartbeater.cc:511] Master 127.13.157.190:44603 requested a full tablet report, sending...
I20260925 02:47:01.399914 348 ts_manager.cc:194] Registered new tserver with Master: 7a8dbdf4dc9547abb9bcd3f62ffdaf54 (127.13.157.129:44363)
I20260925 02:47:01.401530 348 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.157.129:44887
W20260925 02:47:01.499788 540 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:47:01.500078 540 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:47:01.500118 540 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:47:01.504141 540 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:47:01.504194 540 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:47:01.504247 540 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.157.130
I20260925 02:47:01.509047 540 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:39171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790304178325258-13942-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790304178325258-13942-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.157.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790304178325258-13942-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.157.130
--webserver_port=0
--tserver_master_addrs=127.13.157.190:44603
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.540
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.157.130
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790304178325258-13942-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:47:01.510097 540 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:47:01.511476 540 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:47:01.520869 545 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:47:01.520962 546 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:47:01.520944 540 server_base.cc:1034] running on GCE node
W20260925 02:47:01.520886 548 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:47:01.521584 540 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:47:01.522228 540 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:47:01.523380 540 hybrid_clock.cc:648] HybridClock initialized: now 1790304421523363 us; error 30 us; skew 500 ppm
I20260925 02:47:01.525192 540 webserver.cc:460] Webserver started at http://127.13.157.130:36729/ using document root <none> and password file <none>
I20260925 02:47:01.525733 540 fs_manager.cc:359] Metadata directory not provided
I20260925 02:47:01.525791 540 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:47:01.525969 540 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:47:01.528141 540 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790304178325258-13942-0/minicluster-data/ts-1/data/instance:
uuid: "2836e09aa51449219af3e7bed58ab0b7"
format_stamp: "Formatted at 2026-09-25 02:47:01 on dist-test-slave-kznw"
server_key: "183be92cb529cb11a4f5aafb5defb36f"
server_key_iv: "12a86bc15f0d5924280166c87a9f6266"
server_key_version: "encryptionkey@0"
I20260925 02:47:01.528679 540 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790304178325258-13942-0/minicluster-data/ts-1/wal/instance:
uuid: "2836e09aa51449219af3e7bed58ab0b7"
format_stamp: "Formatted at 2026-09-25 02:47:01 on dist-test-slave-kznw"
server_key: "183be92cb529cb11a4f5aafb5defb36f"
server_key_iv: "12a86bc15f0d5924280166c87a9f6266"
server_key_version: "encryptionkey@0"
I20260925 02:47:01.532357 540 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260925 02:47:01.534835 554 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:47:01.536254 540 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 02:47:01.536356 540 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790304178325258-13942-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790304178325258-13942-0/minicluster-data/ts-1/wal
uuid: "2836e09aa51449219af3e7bed58ab0b7"
format_stamp: "Formatted at 2026-09-25 02:47:01 on dist-test-slave-kznw"
server_key: "183be92cb529cb11a4f5aafb5defb36f"
server_key_iv: "12a86bc15f0d5924280166c87a9f6266"
server_key_version: "encryptionkey@0"
I20260925 02:47:01.536463 540 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790304178325258-13942-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790304178325258-13942-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790304178325258-13942-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:47:01.557165 540 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:47:01.557904 540 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:47:01.558103 540 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:47:01.558871 540 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:47:01.560087 540 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:47:01.560143 540 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:47:01.560199 540 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:47:01.560230 540 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:47:01.568873 540 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.130:44031
I20260925 02:47:01.568931 667 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.130:44031 every 8 connection(s)
I20260925 02:47:01.569796 540 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790304178325258-13942-0/minicluster-data/ts-1/data/info.pb
I20260925 02:47:01.575281 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 540
I20260925 02:47:01.575398 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790304178325258-13942-0/minicluster-data/ts-1/wal/instance
I20260925 02:47:01.575769 13942 external_mini_cluster.cc:1428] Setting key 3211c3069f03e13b8edf80d177c59945
I20260925 02:47:01.577778 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790304178325258-13942-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790304178325258-13942-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790304178325258-13942-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790304178325258-13942-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.157.131:0
--local_ip_for_outbound_sockets=127.13.157.131
--webserver_interface=127.13.157.131
--webserver_port=0
--tserver_master_addrs=127.13.157.190:44603
--builtin_ntp_servers=127.13.157.148:39171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260925 02:47:01.579466 388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:01.571637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.130:51315 (local address 127.13.157.190:44603)
0925 02:47:01.571806 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:01.571811 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:01.572902 (+ 1091us) server_negotiation.cc:407] Connection header received
0925 02:47:01.574266 (+ 1364us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:01.574268 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:01.574318 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:01.574391 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:47:01.575338 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:01.576103 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:01.577414 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:01.577726 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:01.577836 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:01.578943 (+ 1107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:47:01.578948 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:47:01.578950 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:47:01.578966 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:47:01.578989 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:47:01.579001 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:47:01.579105 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:01.579302 (+ 197us) server_negotiation.cc:299] Negotiation successful
0925 02:47:01.579355 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260925 02:47:01.580559 670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:01.572074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:44603 (local address 127.13.157.130:51315)
0925 02:47:01.572753 (+ 679us) negotiation.cc:107] Waiting for socket to connect
0925 02:47:01.572795 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0925 02:47:01.574065 (+ 1270us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:47:01.574548 (+ 483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:47:01.574554 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:47:01.574642 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:47:01.575196 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:47:01.575210 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:47:01.576239 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:47:01.576243 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:47:01.577305 (+ 1062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:47:01.577316 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:47:01.577866 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:47:01.577870 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:47:01.578065 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:01.578654 (+ 589us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 02:47:01.578655 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:47:01.578699 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 02:47:01.578765 (+ 66us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:47:01.579142 (+ 377us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:47:01.579153 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:47:01.579160 (+ 7us) client_negotiation.cc:769] Sending connection context
0925 02:47:01.579299 (+ 139us) client_negotiation.cc:240] Negotiation successful
0925 02:47:01.579598 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":114,"threads_started":1}
I20260925 02:47:01.581597 668 heartbeater.cc:347] Connected to a master server at 127.13.157.190:44603
I20260925 02:47:01.581844 668 heartbeater.cc:464] Registering TS with master...
I20260925 02:47:01.582476 668 heartbeater.cc:511] Master 127.13.157.190:44603 requested a full tablet report, sending...
I20260925 02:47:01.583556 348 ts_manager.cc:194] Registered new tserver with Master: 2836e09aa51449219af3e7bed58ab0b7 (127.13.157.130:44031)
I20260925 02:47:01.584107 348 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.157.130:51315
W20260925 02:47:01.690634 671 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:47:01.690922 671 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:47:01.690979 671 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:47:01.695026 671 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:47:01.695075 671 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:47:01.695135 671 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.157.131
I20260925 02:47:01.699978 671 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:39171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790304178325258-13942-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790304178325258-13942-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.157.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790304178325258-13942-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.157.131
--webserver_port=0
--tserver_master_addrs=127.13.157.190:44603
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.671
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.157.131
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790304178325258-13942-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:47:01.701136 671 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:47:01.702556 671 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:47:01.712209 676 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:47:01.712275 671 server_base.cc:1034] running on GCE node
W20260925 02:47:01.712203 677 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:47:01.712198 679 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:47:01.712854 671 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:47:01.713516 671 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:47:01.714689 671 hybrid_clock.cc:648] HybridClock initialized: now 1790304421714667 us; error 38 us; skew 500 ppm
I20260925 02:47:01.716461 671 webserver.cc:460] Webserver started at http://127.13.157.131:33047/ using document root <none> and password file <none>
I20260925 02:47:01.717006 671 fs_manager.cc:359] Metadata directory not provided
I20260925 02:47:01.717077 671 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:47:01.717263 671 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:47:01.719319 671 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790304178325258-13942-0/minicluster-data/ts-2/data/instance:
uuid: "bc58d0486d1745a7af63e85484ad13a3"
format_stamp: "Formatted at 2026-09-25 02:47:01 on dist-test-slave-kznw"
server_key: "744ebc44113ac62d00dc3c0087047e11"
server_key_iv: "aed7742455c8bac8b9dcbf8d102c2943"
server_key_version: "encryptionkey@0"
I20260925 02:47:01.719856 671 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790304178325258-13942-0/minicluster-data/ts-2/wal/instance:
uuid: "bc58d0486d1745a7af63e85484ad13a3"
format_stamp: "Formatted at 2026-09-25 02:47:01 on dist-test-slave-kznw"
server_key: "744ebc44113ac62d00dc3c0087047e11"
server_key_iv: "aed7742455c8bac8b9dcbf8d102c2943"
server_key_version: "encryptionkey@0"
I20260925 02:47:01.723322 671 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260925 02:47:01.725749 685 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:47:01.726970 671 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260925 02:47:01.727072 671 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790304178325258-13942-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790304178325258-13942-0/minicluster-data/ts-2/wal
uuid: "bc58d0486d1745a7af63e85484ad13a3"
format_stamp: "Formatted at 2026-09-25 02:47:01 on dist-test-slave-kznw"
server_key: "744ebc44113ac62d00dc3c0087047e11"
server_key_iv: "aed7742455c8bac8b9dcbf8d102c2943"
server_key_version: "encryptionkey@0"
I20260925 02:47:01.727164 671 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790304178325258-13942-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790304178325258-13942-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790304178325258-13942-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:47:01.736976 671 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:47:01.737664 671 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:47:01.737825 671 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:47:01.738440 671 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:47:01.739467 671 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:47:01.739528 671 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:47:01.739584 671 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:47:01.739616 671 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:47:01.747709 671 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.131:36693
I20260925 02:47:01.747746 798 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.131:36693 every 8 connection(s)
I20260925 02:47:01.748694 671 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790304178325258-13942-0/minicluster-data/ts-2/data/info.pb
I20260925 02:47:01.754354 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 671
I20260925 02:47:01.754441 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790304178325258-13942-0/minicluster-data/ts-2/wal/instance
I20260925 02:47:01.754703 13942 external_mini_cluster.cc:1428] Setting key 5e64966e3b10ec072af6162aad2e543b
I20260925 02:47:01.757927 388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:01.750499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.131:35899 (local address 127.13.157.190:44603)
0925 02:47:01.750631 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:01.750635 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:01.751801 (+ 1166us) server_negotiation.cc:407] Connection header received
0925 02:47:01.753148 (+ 1347us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:01.753151 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:01.753209 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:01.753282 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:47:01.754166 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:01.754738 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:01.756081 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:01.756305 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:01.756391 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:01.757486 (+ 1095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:47:01.757489 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:47:01.757491 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:47:01.757503 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:47:01.757521 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:47:01.757528 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:47:01.757585 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:01.757731 (+ 146us) server_negotiation.cc:299] Negotiation successful
0925 02:47:01.757790 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260925 02:47:01.758734 801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:01.750950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:44603 (local address 127.13.157.131:35899)
0925 02:47:01.751652 (+ 702us) negotiation.cc:107] Waiting for socket to connect
0925 02:47:01.751691 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0925 02:47:01.752944 (+ 1253us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:47:01.753455 (+ 511us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:47:01.753462 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:47:01.753554 (+ 92us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:47:01.754029 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:47:01.754041 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:47:01.754850 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:47:01.754853 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:47:01.755968 (+ 1115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:47:01.755978 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:47:01.756437 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:47:01.756440 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:47:01.756618 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:01.757243 (+ 625us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 02:47:01.757245 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:47:01.757287 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 02:47:01.757347 (+ 60us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:47:01.757594 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:47:01.757603 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:47:01.757609 (+ 6us) client_negotiation.cc:769] Sending connection context
0925 02:47:01.757720 (+ 111us) client_negotiation.cc:240] Negotiation successful
0925 02:47:01.757986 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":121,"threads_started":1}
I20260925 02:47:01.759780 799 heartbeater.cc:347] Connected to a master server at 127.13.157.190:44603
I20260925 02:47:01.759970 799 heartbeater.cc:464] Registering TS with master...
I20260925 02:47:01.760445 799 heartbeater.cc:511] Master 127.13.157.190:44603 requested a full tablet report, sending...
I20260925 02:47:01.761483 348 ts_manager.cc:194] Registered new tserver with Master: bc58d0486d1745a7af63e85484ad13a3 (127.13.157.131:36693)
I20260925 02:47:01.762054 348 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.157.131:35899
I20260925 02:47:01.768559 13942 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 02:47:01.773903 388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:01.770384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49168 (local address 127.13.157.190:44603)
0925 02:47:01.770522 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:01.770527 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:01.770690 (+ 163us) server_negotiation.cc:407] Connection header received
0925 02:47:01.770785 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:01.770787 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:01.770826 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:01.770887 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:47:01.771135 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:01.771650 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:01.772454 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:01.772679 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:01.772752 (+ 73us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:01.773450 (+ 698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:47:01.773454 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:47:01.773456 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:47:01.773467 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:47:01.773485 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:47:01.773492 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:47:01.773621 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:01.773723 (+ 102us) server_negotiation.cc:299] Negotiation successful
0925 02:47:01.773759 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260925 02:47:01.776616 348 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49168:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 02:47:01.779640 348 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 02:47:01.793579 813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:01.787839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.129:44363 (local address 127.0.0.1:37568)
0925 02:47:01.788602 (+ 763us) negotiation.cc:107] Waiting for socket to connect
0925 02:47:01.788615 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 02:47:01.788725 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:47:01.789468 (+ 743us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:47:01.789474 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:47:01.789512 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:47:01.789706 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:47:01.789713 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:47:01.790828 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:47:01.790831 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:47:01.792214 (+ 1383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:47:01.792225 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:47:01.793044 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:47:01.793048 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:47:01.793154 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:01.793191 (+ 37us) client_negotiation.cc:769] Sending connection context
0925 02:47:01.793303 (+ 112us) client_negotiation.cc:240] Negotiation successful
0925 02:47:01.793418 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":703,"mutex_wait_us":50,"spinlock_wait_cycles":3712,"thread_start_us":82,"threads_started":1}
I20260925 02:47:01.793582 815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:01.788541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37568 (local address 127.13.157.129:44363)
0925 02:47:01.789010 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:01.789020 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:01.789042 (+ 22us) server_negotiation.cc:407] Connection header received
0925 02:47:01.789116 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:01.789121 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:01.789304 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:01.789853 (+ 549us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:47:01.790005 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:01.790721 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:01.792358 (+ 1637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:01.792919 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:01.793035 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:01.793233 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:01.793334 (+ 101us) server_negotiation.cc:299] Negotiation successful
0925 02:47:01.793432 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":124,"threads_started":1}
I20260925 02:47:01.793922 811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:01.788156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.131:36693 (local address 127.0.0.1:42006)
0925 02:47:01.789202 (+ 1046us) negotiation.cc:107] Waiting for socket to connect
0925 02:47:01.789214 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 02:47:01.789291 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:47:01.789773 (+ 482us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:47:01.789775 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:47:01.789793 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:47:01.789863 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:47:01.789868 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:47:01.791222 (+ 1354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:47:01.791225 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:47:01.792225 (+ 1000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:47:01.792234 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:47:01.793462 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:47:01.793466 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:47:01.793626 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:01.793648 (+ 22us) client_negotiation.cc:769] Sending connection context
0925 02:47:01.793717 (+ 69us) client_negotiation.cc:240] Negotiation successful
0925 02:47:01.793786 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":987,"mutex_wait_us":29,"thread_start_us":93,"threads_started":1}
I20260925 02:47:01.794240 810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:01.787748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.130:44031 (local address 127.0.0.1:48252)
0925 02:47:01.788530 (+ 782us) negotiation.cc:107] Waiting for socket to connect
0925 02:47:01.788590 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0925 02:47:01.788725 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:47:01.789466 (+ 741us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:47:01.789472 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:47:01.789508 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:47:01.789710 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:47:01.789718 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:47:01.791160 (+ 1442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:47:01.791163 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:47:01.792297 (+ 1134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:47:01.792306 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:47:01.793462 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:47:01.793466 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:47:01.793604 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:01.793786 (+ 182us) client_negotiation.cc:769] Sending connection context
0925 02:47:01.793843 (+ 57us) client_negotiation.cc:240] Negotiation successful
0925 02:47:01.793897 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":622,"mutex_wait_us":106,"spinlock_wait_cycles":9856,"thread_start_us":85,"threads_started":1}
I20260925 02:47:01.794407 812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:01.787911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48252 (local address 127.13.157.130:44031)
0925 02:47:01.788390 (+ 479us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:01.788403 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:01.788861 (+ 458us) server_negotiation.cc:407] Connection header received
0925 02:47:01.788946 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:01.788952 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:01.789165 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:01.789311 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:47:01.789828 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:01.791041 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:01.792399 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:01.793125 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:01.793239 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:01.793398 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:01.794074 (+ 676us) server_negotiation.cc:299] Negotiation successful
0925 02:47:01.794179 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":374,"thread_start_us":141,"threads_started":1}
I20260925 02:47:01.794586 814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:01.788562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42006 (local address 127.13.157.131:36693)
0925 02:47:01.788972 (+ 410us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:01.788983 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:01.789382 (+ 399us) server_negotiation.cc:407] Connection header received
0925 02:47:01.789461 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:01.789466 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:01.789640 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:01.789792 (+ 152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:47:01.789968 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:01.791124 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:01.792354 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:01.793125 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:01.793240 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:01.793398 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:01.794247 (+ 849us) server_negotiation.cc:299] Negotiation successful
0925 02:47:01.794427 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":98,"threads_started":1}
I20260925 02:47:01.796752 471 tablet_service.cc:1467] Processing CreateTablet for tablet a069986a2bff4bb49d12c8705e9dd15f (DEFAULT_TABLE table=test-table [id=8855a80e1a7149809740b520f73fc7bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:47:01.797691 471 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a069986a2bff4bb49d12c8705e9dd15f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:47:01.797422 733 tablet_service.cc:1467] Processing CreateTablet for tablet a069986a2bff4bb49d12c8705e9dd15f (DEFAULT_TABLE table=test-table [id=8855a80e1a7149809740b520f73fc7bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:47:01.797930 602 tablet_service.cc:1467] Processing CreateTablet for tablet a069986a2bff4bb49d12c8705e9dd15f (DEFAULT_TABLE table=test-table [id=8855a80e1a7149809740b520f73fc7bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:47:01.798478 733 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a069986a2bff4bb49d12c8705e9dd15f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:47:01.798923 602 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a069986a2bff4bb49d12c8705e9dd15f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:47:01.804757 816 tablet_bootstrap.cc:492] T a069986a2bff4bb49d12c8705e9dd15f P 7a8dbdf4dc9547abb9bcd3f62ffdaf54: Bootstrap starting.
I20260925 02:47:01.805483 818 tablet_bootstrap.cc:492] T a069986a2bff4bb49d12c8705e9dd15f P bc58d0486d1745a7af63e85484ad13a3: Bootstrap starting.
I20260925 02:47:01.807204 817 tablet_bootstrap.cc:492] T a069986a2bff4bb49d12c8705e9dd15f P 2836e09aa51449219af3e7bed58ab0b7: Bootstrap starting.
I20260925 02:47:01.807212 816 tablet_bootstrap.cc:654] T a069986a2bff4bb49d12c8705e9dd15f P 7a8dbdf4dc9547abb9bcd3f62ffdaf54: Neither blocks nor log segments found. Creating new log.
I20260925 02:47:01.807643 818 tablet_bootstrap.cc:654] T a069986a2bff4bb49d12c8705e9dd15f P bc58d0486d1745a7af63e85484ad13a3: Neither blocks nor log segments found. Creating new log.
I20260925 02:47:01.808194 816 log.cc:824] T a069986a2bff4bb49d12c8705e9dd15f P 7a8dbdf4dc9547abb9bcd3f62ffdaf54: Log is configured to *not* fsync() on all Append() calls
I20260925 02:47:01.808400 818 log.cc:824] T a069986a2bff4bb49d12c8705e9dd15f P bc58d0486d1745a7af63e85484ad13a3: Log is configured to *not* fsync() on all Append() calls
I20260925 02:47:01.808957 817 tablet_bootstrap.cc:654] T a069986a2bff4bb49d12c8705e9dd15f P 2836e09aa51449219af3e7bed58ab0b7: Neither blocks nor log segments found. Creating new log.
I20260925 02:47:01.809676 817 log.cc:824] T a069986a2bff4bb49d12c8705e9dd15f P 2836e09aa51449219af3e7bed58ab0b7: Log is configured to *not* fsync() on all Append() calls
I20260925 02:47:01.810308 816 tablet_bootstrap.cc:492] T a069986a2bff4bb49d12c8705e9dd15f P 7a8dbdf4dc9547abb9bcd3f62ffdaf54: No bootstrap required, opened a new log
I20260925 02:47:01.810510 816 ts_tablet_manager.cc:1397] T a069986a2bff4bb49d12c8705e9dd15f P 7a8dbdf4dc9547abb9bcd3f62ffdaf54: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260925 02:47:01.810532 818 tablet_bootstrap.cc:492] T a069986a2bff4bb49d12c8705e9dd15f P bc58d0486d1745a7af63e85484ad13a3: No bootstrap required, opened a new log
I20260925 02:47:01.810683 818 ts_tablet_manager.cc:1397] T a069986a2bff4bb49d12c8705e9dd15f P bc58d0486d1745a7af63e85484ad13a3: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260925 02:47:01.811340 817 tablet_bootstrap.cc:492] T a069986a2bff4bb49d12c8705e9dd15f P 2836e09aa51449219af3e7bed58ab0b7: No bootstrap required, opened a new log
I20260925 02:47:01.811504 817 ts_tablet_manager.cc:1397] T a069986a2bff4bb49d12c8705e9dd15f P 2836e09aa51449219af3e7bed58ab0b7: Time spent bootstrapping tablet: real 0.004s user 0.004s sys 0.000s
I20260925 02:47:01.814424 817 raft_consensus.cc:348] T a069986a2bff4bb49d12c8705e9dd15f P 2836e09aa51449219af3e7bed58ab0b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2836e09aa51449219af3e7bed58ab0b7" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 44031 } } peers { permanent_uuid: "7a8dbdf4dc9547abb9bcd3f62ffdaf54" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 44363 } } peers { permanent_uuid: "bc58d0486d1745a7af63e85484ad13a3" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 36693 } }
I20260925 02:47:01.814642 817 raft_consensus.cc:374] T a069986a2bff4bb49d12c8705e9dd15f P 2836e09aa51449219af3e7bed58ab0b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:47:01.814693 817 raft_consensus.cc:729] T a069986a2bff4bb49d12c8705e9dd15f P 2836e09aa51449219af3e7bed58ab0b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2836e09aa51449219af3e7bed58ab0b7, State: Initialized, Role: FOLLOWER
I20260925 02:47:01.814716 816 raft_consensus.cc:348] T a069986a2bff4bb49d12c8705e9dd15f P 7a8dbdf4dc9547abb9bcd3f62ffdaf54 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2836e09aa51449219af3e7bed58ab0b7" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 44031 } } peers { permanent_uuid: "7a8dbdf4dc9547abb9bcd3f62ffdaf54" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 44363 } } peers { permanent_uuid: "bc58d0486d1745a7af63e85484ad13a3" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 36693 } }
I20260925 02:47:01.814716 818 raft_consensus.cc:348] T a069986a2bff4bb49d12c8705e9dd15f P bc58d0486d1745a7af63e85484ad13a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2836e09aa51449219af3e7bed58ab0b7" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 44031 } } peers { permanent_uuid: "7a8dbdf4dc9547abb9bcd3f62ffdaf54" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 44363 } } peers { permanent_uuid: "bc58d0486d1745a7af63e85484ad13a3" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 36693 } }
I20260925 02:47:01.815004 818 raft_consensus.cc:374] T a069986a2bff4bb49d12c8705e9dd15f P bc58d0486d1745a7af63e85484ad13a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:47:01.814996 816 raft_consensus.cc:374] T a069986a2bff4bb49d12c8705e9dd15f P 7a8dbdf4dc9547abb9bcd3f62ffdaf54 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:47:01.815063 818 raft_consensus.cc:729] T a069986a2bff4bb49d12c8705e9dd15f P bc58d0486d1745a7af63e85484ad13a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc58d0486d1745a7af63e85484ad13a3, State: Initialized, Role: FOLLOWER
I20260925 02:47:01.815064 816 raft_consensus.cc:729] T a069986a2bff4bb49d12c8705e9dd15f P 7a8dbdf4dc9547abb9bcd3f62ffdaf54 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a8dbdf4dc9547abb9bcd3f62ffdaf54, State: Initialized, Role: FOLLOWER
I20260925 02:47:01.815137 817 consensus_queue.cc:260] T a069986a2bff4bb49d12c8705e9dd15f P 2836e09aa51449219af3e7bed58ab0b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2836e09aa51449219af3e7bed58ab0b7" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 44031 } } peers { permanent_uuid: "7a8dbdf4dc9547abb9bcd3f62ffdaf54" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 44363 } } peers { permanent_uuid: "bc58d0486d1745a7af63e85484ad13a3" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 36693 } }
I20260925 02:47:01.815524 818 consensus_queue.cc:260] T a069986a2bff4bb49d12c8705e9dd15f P bc58d0486d1745a7af63e85484ad13a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2836e09aa51449219af3e7bed58ab0b7" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 44031 } } peers { permanent_uuid: "7a8dbdf4dc9547abb9bcd3f62ffdaf54" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 44363 } } peers { permanent_uuid: "bc58d0486d1745a7af63e85484ad13a3" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 36693 } }
I20260925 02:47:01.815526 816 consensus_queue.cc:260] T a069986a2bff4bb49d12c8705e9dd15f P 7a8dbdf4dc9547abb9bcd3f62ffdaf54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2836e09aa51449219af3e7bed58ab0b7" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 44031 } } peers { permanent_uuid: "7a8dbdf4dc9547abb9bcd3f62ffdaf54" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 44363 } } peers { permanent_uuid: "bc58d0486d1745a7af63e85484ad13a3" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 36693 } }
I20260925 02:47:01.815930 668 heartbeater.cc:503] Master 127.13.157.190:44603 was elected leader, sending a full tablet report...
I20260925 02:47:01.816120 817 ts_tablet_manager.cc:1428] T a069986a2bff4bb49d12c8705e9dd15f P 2836e09aa51449219af3e7bed58ab0b7: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260925 02:47:01.816478 818 ts_tablet_manager.cc:1428] T a069986a2bff4bb49d12c8705e9dd15f P bc58d0486d1745a7af63e85484ad13a3: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260925 02:47:01.816478 816 ts_tablet_manager.cc:1428] T a069986a2bff4bb49d12c8705e9dd15f P 7a8dbdf4dc9547abb9bcd3f62ffdaf54: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260925 02:47:01.816819 799 heartbeater.cc:503] Master 127.13.157.190:44603 was elected leader, sending a full tablet report...
I20260925 02:47:01.816825 537 heartbeater.cc:503] Master 127.13.157.190:44603 was elected leader, sending a full tablet report...
W20260925 02:47:01.820665 669 tablet.cc:2369] T a069986a2bff4bb49d12c8705e9dd15f P 2836e09aa51449219af3e7bed58ab0b7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 02:47:01.886235 538 tablet.cc:2369] T a069986a2bff4bb49d12c8705e9dd15f P 7a8dbdf4dc9547abb9bcd3f62ffdaf54: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 02:47:01.952327 824 raft_consensus.cc:482] T a069986a2bff4bb49d12c8705e9dd15f P bc58d0486d1745a7af63e85484ad13a3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 02:47:01.952534 824 raft_consensus.cc:504] T a069986a2bff4bb49d12c8705e9dd15f P bc58d0486d1745a7af63e85484ad13a3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2836e09aa51449219af3e7bed58ab0b7" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 44031 } } peers { permanent_uuid: "7a8dbdf4dc9547abb9bcd3f62ffdaf54" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 44363 } } peers { permanent_uuid: "bc58d0486d1745a7af63e85484ad13a3" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 36693 } }
I20260925 02:47:01.953717 824 leader_election.cc:288] T a069986a2bff4bb49d12c8705e9dd15f P bc58d0486d1745a7af63e85484ad13a3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2836e09aa51449219af3e7bed58ab0b7 (127.13.157.130:44031), 7a8dbdf4dc9547abb9bcd3f62ffdaf54 (127.13.157.129:44363)
I20260925 02:47:01.957620 825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:01.954008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.130:44031 (local address 127.13.157.131:44899)
0925 02:47:01.954440 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0925 02:47:01.954454 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 02:47:01.954572 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:47:01.954816 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:47:01.954818 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:47:01.954832 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:47:01.954962 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:47:01.954970 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:47:01.955736 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:47:01.955738 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:47:01.956574 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:47:01.956583 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:47:01.957193 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:47:01.957197 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:47:01.957322 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:01.957347 (+ 25us) client_negotiation.cc:769] Sending connection context
0925 02:47:01.957406 (+ 59us) client_negotiation.cc:240] Negotiation successful
0925 02:47:01.957460 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":369,"mutex_wait_us":63,"thread_start_us":145,"threads_started":1}
I20260925 02:47:01.957649 812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:01.954101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.131:44899 (local address 127.13.157.130:44031)
0925 02:47:01.954247 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:01.954252 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:01.954555 (+ 303us) server_negotiation.cc:407] Connection header received
0925 02:47:01.954669 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:01.954671 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:01.954725 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:01.954798 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:47:01.955081 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:01.955628 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:01.956689 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:01.957101 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:01.957192 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:01.957240 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:01.957441 (+ 201us) server_negotiation.cc:299] Negotiation successful
0925 02:47:01.957520 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260925 02:47:01.957697 815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:01.954089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.131:38963 (local address 127.13.157.129:44363)
0925 02:47:01.954242 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:01.954247 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:01.954469 (+ 222us) server_negotiation.cc:407] Connection header received
0925 02:47:01.954570 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:01.954573 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:01.954634 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:01.954700 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:47:01.954992 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:01.955535 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:01.956706 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:01.957153 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:01.957268 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:01.957315 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:01.957515 (+ 200us) server_negotiation.cc:299] Negotiation successful
0925 02:47:01.957583 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260925 02:47:01.957659 801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:01.954008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.129:44363 (local address 127.13.157.131:38963)
0925 02:47:01.954384 (+ 376us) negotiation.cc:107] Waiting for socket to connect
0925 02:47:01.954397 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 02:47:01.954485 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:47:01.954721 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:47:01.954724 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:47:01.954739 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:47:01.954859 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:47:01.954867 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:47:01.955629 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:47:01.955632 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:47:01.956572 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:47:01.956581 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:47:01.957275 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:47:01.957278 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:47:01.957389 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:01.957412 (+ 23us) client_negotiation.cc:769] Sending connection context
0925 02:47:01.957485 (+ 73us) client_negotiation.cc:240] Negotiation successful
0925 02:47:01.957556 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":310,"mutex_wait_us":27}
I20260925 02:47:01.958622 491 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a069986a2bff4bb49d12c8705e9dd15f" candidate_uuid: "bc58d0486d1745a7af63e85484ad13a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a8dbdf4dc9547abb9bcd3f62ffdaf54" is_pre_election: true
I20260925 02:47:01.958624 622 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a069986a2bff4bb49d12c8705e9dd15f" candidate_uuid: "bc58d0486d1745a7af63e85484ad13a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2836e09aa51449219af3e7bed58ab0b7" is_pre_election: true
I20260925 02:47:01.958931 491 raft_consensus.cc:2454] T a069986a2bff4bb49d12c8705e9dd15f P 7a8dbdf4dc9547abb9bcd3f62ffdaf54 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc58d0486d1745a7af63e85484ad13a3 in term 0.
I20260925 02:47:01.958940 622 raft_consensus.cc:2454] T a069986a2bff4bb49d12c8705e9dd15f P 2836e09aa51449219af3e7bed58ab0b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc58d0486d1745a7af63e85484ad13a3 in term 0.
I20260925 02:47:01.959407 686 leader_election.cc:302] T a069986a2bff4bb49d12c8705e9dd15f P bc58d0486d1745a7af63e85484ad13a3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2836e09aa51449219af3e7bed58ab0b7, bc58d0486d1745a7af63e85484ad13a3; no voters:
I20260925 02:47:01.959671 824 raft_consensus.cc:2781] T a069986a2bff4bb49d12c8705e9dd15f P bc58d0486d1745a7af63e85484ad13a3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 02:47:01.959751 824 raft_consensus.cc:482] T a069986a2bff4bb49d12c8705e9dd15f P bc58d0486d1745a7af63e85484ad13a3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 02:47:01.959782 824 raft_consensus.cc:3037] T a069986a2bff4bb49d12c8705e9dd15f P bc58d0486d1745a7af63e85484ad13a3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:47:01.960757 824 raft_consensus.cc:504] T a069986a2bff4bb49d12c8705e9dd15f P bc58d0486d1745a7af63e85484ad13a3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2836e09aa51449219af3e7bed58ab0b7" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 44031 } } peers { permanent_uuid: "7a8dbdf4dc9547abb9bcd3f62ffdaf54" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 44363 } } peers { permanent_uuid: "bc58d0486d1745a7af63e85484ad13a3" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 36693 } }
I20260925 02:47:01.961086 824 leader_election.cc:288] T a069986a2bff4bb49d12c8705e9dd15f P bc58d0486d1745a7af63e85484ad13a3 [CANDIDATE]: Term 1 election: Requested vote from peers 2836e09aa51449219af3e7bed58ab0b7 (127.13.157.130:44031), 7a8dbdf4dc9547abb9bcd3f62ffdaf54 (127.13.157.129:44363)
I20260925 02:47:01.961395 491 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a069986a2bff4bb49d12c8705e9dd15f" candidate_uuid: "bc58d0486d1745a7af63e85484ad13a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a8dbdf4dc9547abb9bcd3f62ffdaf54"
I20260925 02:47:01.961503 491 raft_consensus.cc:3037] T a069986a2bff4bb49d12c8705e9dd15f P 7a8dbdf4dc9547abb9bcd3f62ffdaf54 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:47:01.962518 491 raft_consensus.cc:2454] T a069986a2bff4bb49d12c8705e9dd15f P 7a8dbdf4dc9547abb9bcd3f62ffdaf54 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc58d0486d1745a7af63e85484ad13a3 in term 1.
I20260925 02:47:01.962778 622 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a069986a2bff4bb49d12c8705e9dd15f" candidate_uuid: "bc58d0486d1745a7af63e85484ad13a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2836e09aa51449219af3e7bed58ab0b7"
I20260925 02:47:01.962819 687 leader_election.cc:302] T a069986a2bff4bb49d12c8705e9dd15f P bc58d0486d1745a7af63e85484ad13a3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a8dbdf4dc9547abb9bcd3f62ffdaf54, bc58d0486d1745a7af63e85484ad13a3; no voters:
I20260925 02:47:01.962942 622 raft_consensus.cc:3037] T a069986a2bff4bb49d12c8705e9dd15f P 2836e09aa51449219af3e7bed58ab0b7 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:47:01.963001 824 raft_consensus.cc:2781] T a069986a2bff4bb49d12c8705e9dd15f P bc58d0486d1745a7af63e85484ad13a3 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:47:01.963228 824 raft_consensus.cc:686] T a069986a2bff4bb49d12c8705e9dd15f P bc58d0486d1745a7af63e85484ad13a3 [term 1 LEADER]: Becoming Leader. State: Replica: bc58d0486d1745a7af63e85484ad13a3, State: Running, Role: LEADER
I20260925 02:47:01.963558 824 consensus_queue.cc:237] T a069986a2bff4bb49d12c8705e9dd15f P bc58d0486d1745a7af63e85484ad13a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2836e09aa51449219af3e7bed58ab0b7" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 44031 } } peers { permanent_uuid: "7a8dbdf4dc9547abb9bcd3f62ffdaf54" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 44363 } } peers { permanent_uuid: "bc58d0486d1745a7af63e85484ad13a3" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 36693 } }
I20260925 02:47:01.964053 622 raft_consensus.cc:2454] T a069986a2bff4bb49d12c8705e9dd15f P 2836e09aa51449219af3e7bed58ab0b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc58d0486d1745a7af63e85484ad13a3 in term 1.
I20260925 02:47:01.966857 348 catalog_manager.cc:5495] T a069986a2bff4bb49d12c8705e9dd15f P bc58d0486d1745a7af63e85484ad13a3 reported cstate change: term changed from 0 to 1, leader changed from <none> to bc58d0486d1745a7af63e85484ad13a3 (127.13.157.131). New cstate: current_term: 1 leader_uuid: "bc58d0486d1745a7af63e85484ad13a3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2836e09aa51449219af3e7bed58ab0b7" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 44031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a8dbdf4dc9547abb9bcd3f62ffdaf54" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 44363 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc58d0486d1745a7af63e85484ad13a3" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 36693 } health_report { overall_health: HEALTHY } } }
I20260925 02:47:01.983795 814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:01.980781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42012 (local address 127.13.157.131:36693)
0925 02:47:01.980893 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:01.980896 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:01.980909 (+ 13us) server_negotiation.cc:407] Connection header received
0925 02:47:01.980979 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:01.980981 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:01.981044 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:01.981131 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 02:47:01.981314 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:01.981793 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:01.982559 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:01.982760 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:01.982825 (+ 65us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:01.983107 (+ 282us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 02:47:01.983489 (+ 382us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 02:47:01.983620 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:01.983658 (+ 38us) server_negotiation.cc:299] Negotiation successful
0925 02:47:01.983690 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260925 02:47:01.989351 622 raft_consensus.cc:1261] T a069986a2bff4bb49d12c8705e9dd15f P 2836e09aa51449219af3e7bed58ab0b7 [term 1 FOLLOWER]: Refusing update from remote peer bc58d0486d1745a7af63e85484ad13a3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:47:01.989351 491 raft_consensus.cc:1261] T a069986a2bff4bb49d12c8705e9dd15f P 7a8dbdf4dc9547abb9bcd3f62ffdaf54 [term 1 FOLLOWER]: Refusing update from remote peer bc58d0486d1745a7af63e85484ad13a3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:47:01.990028 826 consensus_queue.cc:1035] T a069986a2bff4bb49d12c8705e9dd15f P bc58d0486d1745a7af63e85484ad13a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a8dbdf4dc9547abb9bcd3f62ffdaf54" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 44363 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:47:01.990190 824 consensus_queue.cc:1035] T a069986a2bff4bb49d12c8705e9dd15f P bc58d0486d1745a7af63e85484ad13a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2836e09aa51449219af3e7bed58ab0b7" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 44031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:47:01.996991 831 mvcc.cc:205] Tried to move back new op lower bound from 7333086912461504512 to 7333086912365387776. Current Snapshot: MvccSnapshot[applied={T|T < 7333086912461504512}]
I20260925 02:47:01.996454 829 mvcc.cc:205] Tried to move back new op lower bound from 7333086912461504512 to 7333086912365387776. Current Snapshot: MvccSnapshot[applied={T|T < 7333086912461504512}]
I20260925 02:47:02.006212 346 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49168:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 02:47:02.006367 346 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49168:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 02:47:02.012605 346 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7499a16eaeaf4ab797b137a223d937d7: Sending DeleteTablet for 3 replicas of tablet a069986a2bff4bb49d12c8705e9dd15f
I20260925 02:47:02.013350 602 tablet_service.cc:1514] Processing DeleteTablet for tablet a069986a2bff4bb49d12c8705e9dd15f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:47:02 UTC) from {username='slave'} at 127.0.0.1:48252
I20260925 02:47:02.013876 839 tablet_replica.cc:330] T a069986a2bff4bb49d12c8705e9dd15f P 2836e09aa51449219af3e7bed58ab0b7: stopping tablet replica
I20260925 02:47:02.014225 839 raft_consensus.cc:2229] T a069986a2bff4bb49d12c8705e9dd15f P 2836e09aa51449219af3e7bed58ab0b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 02:47:02.014576 839 raft_consensus.cc:2258] T a069986a2bff4bb49d12c8705e9dd15f P 2836e09aa51449219af3e7bed58ab0b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:47:02.014751 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 409
I20260925 02:47:02.015812 733 tablet_service.cc:1514] Processing DeleteTablet for tablet a069986a2bff4bb49d12c8705e9dd15f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:47:02 UTC) from {username='slave'} at 127.0.0.1:42006
I20260925 02:47:02.017463 839 ts_tablet_manager.cc:1905] T a069986a2bff4bb49d12c8705e9dd15f P 2836e09aa51449219af3e7bed58ab0b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:47:02.018786 840 tablet_replica.cc:330] T a069986a2bff4bb49d12c8705e9dd15f P bc58d0486d1745a7af63e85484ad13a3: stopping tablet replica
I20260925 02:47:02.019145 840 raft_consensus.cc:2229] T a069986a2bff4bb49d12c8705e9dd15f P bc58d0486d1745a7af63e85484ad13a3 [term 1 LEADER]: Raft consensus shutting down.
I20260925 02:47:02.019585 840 raft_consensus.cc:2258] T a069986a2bff4bb49d12c8705e9dd15f P bc58d0486d1745a7af63e85484ad13a3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:47:02.020370 839 ts_tablet_manager.cc:1918] T a069986a2bff4bb49d12c8705e9dd15f P 2836e09aa51449219af3e7bed58ab0b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 02:47:02.020466 839 log.cc:1196] T a069986a2bff4bb49d12c8705e9dd15f P 2836e09aa51449219af3e7bed58ab0b7: Deleting WAL directory at /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790304178325258-13942-0/minicluster-data/ts-1/wal/wals/a069986a2bff4bb49d12c8705e9dd15f
I20260925 02:47:02.020769 839 ts_tablet_manager.cc:1939] T a069986a2bff4bb49d12c8705e9dd15f P 2836e09aa51449219af3e7bed58ab0b7: Deleting consensus metadata
I20260925 02:47:02.020965 840 ts_tablet_manager.cc:1905] T a069986a2bff4bb49d12c8705e9dd15f P bc58d0486d1745a7af63e85484ad13a3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:47:02.022024 332 catalog_manager.cc:4841] TS 2836e09aa51449219af3e7bed58ab0b7 (127.13.157.130:44031): tablet a069986a2bff4bb49d12c8705e9dd15f (table test-table [id=8855a80e1a7149809740b520f73fc7bc]) successfully deleted
I20260925 02:47:02.023556 840 ts_tablet_manager.cc:1918] T a069986a2bff4bb49d12c8705e9dd15f P bc58d0486d1745a7af63e85484ad13a3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 02:47:02.023655 840 log.cc:1196] T a069986a2bff4bb49d12c8705e9dd15f P bc58d0486d1745a7af63e85484ad13a3: Deleting WAL directory at /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790304178325258-13942-0/minicluster-data/ts-2/wal/wals/a069986a2bff4bb49d12c8705e9dd15f
I20260925 02:47:02.023937 840 ts_tablet_manager.cc:1939] T a069986a2bff4bb49d12c8705e9dd15f P bc58d0486d1745a7af63e85484ad13a3: Deleting consensus metadata
I20260925 02:47:02.025023 334 catalog_manager.cc:4841] TS bc58d0486d1745a7af63e85484ad13a3 (127.13.157.131:36693): tablet a069986a2bff4bb49d12c8705e9dd15f (table test-table [id=8855a80e1a7149809740b520f73fc7bc]) successfully deleted
W20260925 02:47:02.025305 333 connection.cc:535] client connection to 127.13.157.129:44363 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260925 02:47:02.025456 333 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260925 02:47:02.025522 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 540
I20260925 02:47:02.030712 810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:02.030405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.129:44363 (local address 127.0.0.1:37572)
0925 02:47:02.030540 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0925 02:47:02.030617 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.129:44363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260925 02:47:02.030933 333 catalog_manager.cc:4584] TS 7a8dbdf4dc9547abb9bcd3f62ffdaf54 (127.13.157.129:44363): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a069986a2bff4bb49d12c8705e9dd15f: Network error: Client connection negotiation failed: client connection to 127.13.157.129:44363: connect: Connection refused (error 111)
I20260925 02:47:02.032013 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 671
I20260925 02:47:02.036998 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 317
2026-09-25T02:47:02Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1116 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790304178325258-13942-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 02:47:02 dist-test-slave-kznw krb5kdc[844](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 02:47:02 dist-test-slave-kznw krb5kdc[844](info): set up 2 sockets
Sep 25 02:47:02 dist-test-slave-kznw krb5kdc[844](info): commencing operation
krb5kdc: starting...
W20260925 02:47:02.998340 13942 mini_kdc.cc:121] Time spent starting KDC: real 0.941s user 0.004s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790304178325258-13942-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790304178325258-13942-0/krb5kdc/test-user.keytab.
Sep 25 02:47:03 dist-test-slave-kznw krb5kdc[844](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304423, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T02:47:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T02:47:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.157.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.157.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.157.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab.
Entry for principal kudu/127.13.157.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab.
I20260925 02:47:03.115458 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790304178325258-13942-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790304178325258-13942-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790304178325258-13942-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.157.190:44173
--webserver_interface=127.13.157.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab
--principal=kudu/127.13.157.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:34181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.157.190:44173
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:47:03.224207 857 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:47:03.224488 857 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:47:03.224532 857 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:47:03.228572 857 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:47:03.228629 857 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:47:03.228646 857 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:47:03.228662 857 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:47:03.228675 857 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:47:03.233911 857 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:34181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790304178325258-13942-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790304178325258-13942-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.157.190:44173
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab
--principal=kudu/127.13.157.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.157.190:44173
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.157.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.857
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790304178325258-13942-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:47:03.235016 857 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:47:03.236060 857 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:47:03.244985 862 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:47:03.244985 865 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:47:03.244979 863 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:47:03.245200 857 server_base.cc:1034] running on GCE node
I20260925 02:47:03.245915 857 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:47:03.247273 857 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:47:03.248509 857 hybrid_clock.cc:648] HybridClock initialized: now 1790304423248495 us; error 33 us; skew 500 ppm
Sep 25 02:47:03 dist-test-slave-kznw krb5kdc[844](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304423, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:47:03.251359 857 init.cc:377] Logged in from keytab as kudu/127.13.157.190@KRBTEST.COM (short username kudu)
I20260925 02:47:03.252310 857 webserver.cc:460] Webserver started at http://127.13.157.190:43287/ using document root <none> and password file <none>
I20260925 02:47:03.252843 857 fs_manager.cc:359] Metadata directory not provided
I20260925 02:47:03.252904 857 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:47:03.253073 857 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:47:03.255266 857 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790304178325258-13942-0/minicluster-data/master-0/data/instance:
uuid: "6419840f89ab45d5ad6d9350ad4e4220"
format_stamp: "Formatted at 2026-09-25 02:47:03 on dist-test-slave-kznw"
server_key: "0a93abfdfc3a79d1fb6cc7a44ba84697"
server_key_iv: "cac8362711f03949e3a8ad516de59094"
server_key_version: "encryptionkey@0"
I20260925 02:47:03.255959 857 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790304178325258-13942-0/minicluster-data/master-0/wal/instance:
uuid: "6419840f89ab45d5ad6d9350ad4e4220"
format_stamp: "Formatted at 2026-09-25 02:47:03 on dist-test-slave-kznw"
server_key: "0a93abfdfc3a79d1fb6cc7a44ba84697"
server_key_iv: "cac8362711f03949e3a8ad516de59094"
server_key_version: "encryptionkey@0"
I20260925 02:47:03.260046 857 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260925 02:47:03.262646 872 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:47:03.263931 857 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260925 02:47:03.264067 857 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790304178325258-13942-0/minicluster-data/master-0/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790304178325258-13942-0/minicluster-data/master-0/wal
uuid: "6419840f89ab45d5ad6d9350ad4e4220"
format_stamp: "Formatted at 2026-09-25 02:47:03 on dist-test-slave-kznw"
server_key: "0a93abfdfc3a79d1fb6cc7a44ba84697"
server_key_iv: "cac8362711f03949e3a8ad516de59094"
server_key_version: "encryptionkey@0"
I20260925 02:47:03.264174 857 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790304178325258-13942-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790304178325258-13942-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790304178325258-13942-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:47:03.313120 857 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:47:03.316143 857 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:47:03.316324 857 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:47:03.323101 857 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.190:44173
I20260925 02:47:03.323138 924 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.190:44173 every 8 connection(s)
I20260925 02:47:03.324050 857 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
I20260925 02:47:03.326853 925 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:47:03.331754 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 857
I20260925 02:47:03.331871 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790304178325258-13942-0/minicluster-data/master-0/wal/instance
I20260925 02:47:03.332134 13942 external_mini_cluster.cc:1428] Setting key 20b981d7d61053fbd146ed8e61826cbd
I20260925 02:47:03.332403 925 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6419840f89ab45d5ad6d9350ad4e4220: Bootstrap starting.
I20260925 02:47:03.335104 925 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6419840f89ab45d5ad6d9350ad4e4220: Neither blocks nor log segments found. Creating new log.
I20260925 02:47:03.335969 925 log.cc:824] T 00000000000000000000000000000000 P 6419840f89ab45d5ad6d9350ad4e4220: Log is configured to *not* fsync() on all Append() calls
I20260925 02:47:03.337877 925 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6419840f89ab45d5ad6d9350ad4e4220: No bootstrap required, opened a new log
Sep 25 02:47:03 dist-test-slave-kznw krb5kdc[844](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304423, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.157.190@KRBTEST.COM
I20260925 02:47:03.341408 925 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6419840f89ab45d5ad6d9350ad4e4220 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6419840f89ab45d5ad6d9350ad4e4220" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 44173 } }
I20260925 02:47:03.341684 925 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6419840f89ab45d5ad6d9350ad4e4220 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:47:03.341741 925 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6419840f89ab45d5ad6d9350ad4e4220 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6419840f89ab45d5ad6d9350ad4e4220, State: Initialized, Role: FOLLOWER
I20260925 02:47:03.342217 925 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6419840f89ab45d5ad6d9350ad4e4220 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6419840f89ab45d5ad6d9350ad4e4220" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 44173 } }
I20260925 02:47:03.342365 925 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6419840f89ab45d5ad6d9350ad4e4220 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 02:47:03.342427 925 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6419840f89ab45d5ad6d9350ad4e4220 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 02:47:03.342514 925 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6419840f89ab45d5ad6d9350ad4e4220 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:47:03.343354 925 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6419840f89ab45d5ad6d9350ad4e4220 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6419840f89ab45d5ad6d9350ad4e4220" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 44173 } }
I20260925 02:47:03.343668 925 leader_election.cc:302] T 00000000000000000000000000000000 P 6419840f89ab45d5ad6d9350ad4e4220 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6419840f89ab45d5ad6d9350ad4e4220; no voters:
I20260925 02:47:03.343978 925 leader_election.cc:288] T 00000000000000000000000000000000 P 6419840f89ab45d5ad6d9350ad4e4220 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 02:47:03.344043 930 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6419840f89ab45d5ad6d9350ad4e4220 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:47:03.344247 930 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6419840f89ab45d5ad6d9350ad4e4220 [term 1 LEADER]: Becoming Leader. State: Replica: 6419840f89ab45d5ad6d9350ad4e4220, State: Running, Role: LEADER
I20260925 02:47:03.344544 930 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6419840f89ab45d5ad6d9350ad4e4220 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6419840f89ab45d5ad6d9350ad4e4220" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 44173 } }
I20260925 02:47:03.345283 925 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6419840f89ab45d5ad6d9350ad4e4220 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:47:03.346139 931 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6419840f89ab45d5ad6d9350ad4e4220 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6419840f89ab45d5ad6d9350ad4e4220. Latest consensus state: current_term: 1 leader_uuid: "6419840f89ab45d5ad6d9350ad4e4220" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6419840f89ab45d5ad6d9350ad4e4220" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 44173 } } }
I20260925 02:47:03.346269 931 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6419840f89ab45d5ad6d9350ad4e4220 [sys.catalog]: This master's current role is: LEADER
I20260925 02:47:03.346598 932 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6419840f89ab45d5ad6d9350ad4e4220 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6419840f89ab45d5ad6d9350ad4e4220" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6419840f89ab45d5ad6d9350ad4e4220" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 44173 } } }
I20260925 02:47:03.346709 932 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6419840f89ab45d5ad6d9350ad4e4220 [sys.catalog]: This master's current role is: LEADER
I20260925 02:47:03.347178 939 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:47:03.348074 928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:03.333504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43512 (local address 127.13.157.190:44173)
0925 02:47:03.333941 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:03.333960 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:03.334002 (+ 42us) server_negotiation.cc:407] Connection header received
0925 02:47:03.334799 (+ 797us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:03.334822 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:03.335284 (+ 462us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:03.335651 (+ 367us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:47:03.336941 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:03.337988 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:03.338940 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:03.339434 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:03.339596 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:03.342143 (+ 2547us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:47:03.342148 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:47:03.342160 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:47:03.342187 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:47:03.343962 (+ 1775us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:47:03.344478 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:47:03.344485 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:47:03.344491 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:47:03.344632 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:47:03.344929 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:47:03.344933 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:47:03.344935 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:47:03.345331 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:47:03.345558 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:03.346214 (+ 656us) server_negotiation.cc:299] Negotiation successful
0925 02:47:03.346677 (+ 463us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":287,"thread_start_us":118,"threads_started":1}
I20260925 02:47:03.350358 939 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:47:03.356225 939 catalog_manager.cc:1347] Generated new cluster ID: 2d20bd8de6004991b6fb09e3bb9a3658
I20260925 02:47:03.356299 939 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 02:47:03.365217 939 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 02:47:03.366137 939 catalog_manager.cc:1504] Loading token signing keys...
I20260925 02:47:03.370428 939 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6419840f89ab45d5ad6d9350ad4e4220: Generated new TSK 0
I20260925 02:47:03.371258 939 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.157.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.157.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.157.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.129.keytab.
Entry for principal kudu/127.13.157.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.129.keytab.
I20260925 02:47:03.404022 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790304178325258-13942-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790304178325258-13942-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790304178325258-13942-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790304178325258-13942-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.157.129:0
--local_ip_for_outbound_sockets=127.13.157.129
--webserver_interface=127.13.157.129
--webserver_port=0
--tserver_master_addrs=127.13.157.190:44173
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.129.keytab
--principal=kudu/127.13.157.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:34181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:47:03.513540 951 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:47:03.513790 951 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:47:03.513832 951 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:47:03.517800 951 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:47:03.517863 951 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:47:03.517922 951 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.157.129
I20260925 02:47:03.523231 951 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:34181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790304178325258-13942-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790304178325258-13942-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.129.keytab
--principal=kudu/127.13.157.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.157.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790304178325258-13942-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.157.129
--webserver_port=0
--tserver_master_addrs=127.13.157.190:44173
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.951
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.157.129
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790304178325258-13942-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:47:03.524258 951 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:47:03.525280 951 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:47:03.534595 956 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:47:03.534638 959 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:47:03.534605 957 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:47:03.534793 951 server_base.cc:1034] running on GCE node
I20260925 02:47:03.535250 951 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:47:03.535873 951 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:47:03.537034 951 hybrid_clock.cc:648] HybridClock initialized: now 1790304423537016 us; error 32 us; skew 500 ppm
Sep 25 02:47:03 dist-test-slave-kznw krb5kdc[844](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304423, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:47:03.539763 951 init.cc:377] Logged in from keytab as kudu/127.13.157.129@KRBTEST.COM (short username kudu)
I20260925 02:47:03.540578 951 webserver.cc:460] Webserver started at http://127.13.157.129:35001/ using document root <none> and password file <none>
I20260925 02:47:03.541096 951 fs_manager.cc:359] Metadata directory not provided
I20260925 02:47:03.541162 951 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:47:03.541381 951 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:47:03.543543 951 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790304178325258-13942-0/minicluster-data/ts-0/data/instance:
uuid: "3112b67b7cf0405a93b31605ae5e7800"
format_stamp: "Formatted at 2026-09-25 02:47:03 on dist-test-slave-kznw"
server_key: "65acffc5f6a4979ee3f08b63d794acef"
server_key_iv: "6bec94db816d480629f179b9cb70d438"
server_key_version: "encryptionkey@0"
I20260925 02:47:03.544095 951 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790304178325258-13942-0/minicluster-data/ts-0/wal/instance:
uuid: "3112b67b7cf0405a93b31605ae5e7800"
format_stamp: "Formatted at 2026-09-25 02:47:03 on dist-test-slave-kznw"
server_key: "65acffc5f6a4979ee3f08b63d794acef"
server_key_iv: "6bec94db816d480629f179b9cb70d438"
server_key_version: "encryptionkey@0"
I20260925 02:47:03.548028 951 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260925 02:47:03.550668 966 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:47:03.551936 951 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260925 02:47:03.552048 951 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790304178325258-13942-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790304178325258-13942-0/minicluster-data/ts-0/wal
uuid: "3112b67b7cf0405a93b31605ae5e7800"
format_stamp: "Formatted at 2026-09-25 02:47:03 on dist-test-slave-kznw"
server_key: "65acffc5f6a4979ee3f08b63d794acef"
server_key_iv: "6bec94db816d480629f179b9cb70d438"
server_key_version: "encryptionkey@0"
I20260925 02:47:03.552153 951 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790304178325258-13942-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790304178325258-13942-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790304178325258-13942-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:47:03.563500 951 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:47:03.566452 951 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:47:03.566633 951 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:47:03.567343 951 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:47:03.568167 951 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:47:03.568216 951 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:47:03.568266 951 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:47:03.568292 951 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:47:03.576475 951 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.129:37019
I20260925 02:47:03.576577 1079 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.129:37019 every 8 connection(s)
I20260925 02:47:03.577344 951 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790304178325258-13942-0/minicluster-data/ts-0/data/info.pb
I20260925 02:47:03.579790 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 951
I20260925 02:47:03.579921 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790304178325258-13942-0/minicluster-data/ts-0/wal/instance
I20260925 02:47:03.580256 13942 external_mini_cluster.cc:1428] Setting key 4f86d5efdc8ebdb4c9daa149fdbe86c5
Sep 25 02:47:03 dist-test-slave-kznw krb5kdc[844](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304423, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.129@KRBTEST.COM for kudu/127.13.157.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.157.130@KRBTEST.COM; defaulting to no policy
I20260925 02:47:03.591889 928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:03.579108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.129:60977 (local address 127.13.157.190:44173)
0925 02:47:03.579249 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:03.579253 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:03.580177 (+ 924us) server_negotiation.cc:407] Connection header received
0925 02:47:03.581734 (+ 1557us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:03.581738 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:03.581785 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:03.581855 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:47:03.583365 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:03.583879 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:03.585298 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:03.585588 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:03.585686 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:03.588387 (+ 2701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:47:03.588391 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:47:03.588394 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:47:03.588424 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:47:03.589910 (+ 1486us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:47:03.590453 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:47:03.590457 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:47:03.590459 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:47:03.590559 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:47:03.590823 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:47:03.590825 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:47:03.590826 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:47:03.591031 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:47:03.591107 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:03.591655 (+ 548us) server_negotiation.cc:299] Negotiation successful
0925 02:47:03.591762 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.157.130@KRBTEST.COM" created.
I20260925 02:47:03.592777 1082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:03.579472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:44173 (local address 127.13.157.129:60977)
0925 02:47:03.579998 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0925 02:47:03.580057 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0925 02:47:03.581515 (+ 1458us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:47:03.582043 (+ 528us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:47:03.582052 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:47:03.582506 (+ 454us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:47:03.583208 (+ 702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:47:03.583234 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:47:03.584003 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:47:03.584006 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:47:03.585125 (+ 1119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:47:03.585137 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:47:03.585743 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:47:03.585747 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:47:03.585944 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:03.586539 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:47:03.586560 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:47:03.588216 (+ 1656us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:47:03.590049 (+ 1833us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:47:03.590054 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:47:03.590065 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:47:03.590347 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:47:03.590632 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:47:03.590634 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:47:03.590635 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:47:03.590754 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:47:03.591145 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:47:03.591151 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:47:03.591413 (+ 262us) client_negotiation.cc:769] Sending connection context
0925 02:47:03.591576 (+ 163us) client_negotiation.cc:240] Negotiation successful
0925 02:47:03.591835 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":118,"threads_started":1}
I20260925 02:47:03.594059 1080 heartbeater.cc:347] Connected to a master server at 127.13.157.190:44173
I20260925 02:47:03.594306 1080 heartbeater.cc:464] Registering TS with master...
I20260925 02:47:03.594786 1080 heartbeater.cc:511] Master 127.13.157.190:44173 requested a full tablet report, sending...
I20260925 02:47:03.596477 889 ts_manager.cc:194] Registered new tserver with Master: 3112b67b7cf0405a93b31605ae5e7800 (127.13.157.129:37019)
I20260925 02:47:03.598026 889 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.157.129@KRBTEST.COM'} at 127.13.157.129:60977
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.157.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.130.keytab.
Entry for principal kudu/127.13.157.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.130.keytab.
I20260925 02:47:03.609311 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790304178325258-13942-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790304178325258-13942-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790304178325258-13942-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790304178325258-13942-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.157.130:0
--local_ip_for_outbound_sockets=127.13.157.130
--webserver_interface=127.13.157.130
--webserver_port=0
--tserver_master_addrs=127.13.157.190:44173
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.130.keytab
--principal=kudu/127.13.157.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:34181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:47:03.717024 1085 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:47:03.717308 1085 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:47:03.717365 1085 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:47:03.721463 1085 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:47:03.721539 1085 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:47:03.721624 1085 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.157.130
I20260925 02:47:03.726995 1085 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:34181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790304178325258-13942-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790304178325258-13942-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.130.keytab
--principal=kudu/127.13.157.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.157.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790304178325258-13942-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.157.130
--webserver_port=0
--tserver_master_addrs=127.13.157.190:44173
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1085
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.157.130
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790304178325258-13942-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:47:03.728096 1085 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:47:03.729127 1085 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:47:03.739147 1090 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:47:03.739147 1093 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:47:03.739317 1085 server_base.cc:1034] running on GCE node
W20260925 02:47:03.739195 1091 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:47:03.739818 1085 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:47:03.740490 1085 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:47:03.741669 1085 hybrid_clock.cc:648] HybridClock initialized: now 1790304423741638 us; error 40 us; skew 500 ppm
Sep 25 02:47:03 dist-test-slave-kznw krb5kdc[844](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304423, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:47:03.744459 1085 init.cc:377] Logged in from keytab as kudu/127.13.157.130@KRBTEST.COM (short username kudu)
I20260925 02:47:03.745424 1085 webserver.cc:460] Webserver started at http://127.13.157.130:37845/ using document root <none> and password file <none>
I20260925 02:47:03.746032 1085 fs_manager.cc:359] Metadata directory not provided
I20260925 02:47:03.746104 1085 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:47:03.746332 1085 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:47:03.748502 1085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790304178325258-13942-0/minicluster-data/ts-1/data/instance:
uuid: "f74b2f820a3f4588990db88d7a18c341"
format_stamp: "Formatted at 2026-09-25 02:47:03 on dist-test-slave-kznw"
server_key: "02a122a27dbd5c79b6866018383e3fa6"
server_key_iv: "5c7a71bc94df66be8d0e5d3eb62a89d4"
server_key_version: "encryptionkey@0"
I20260925 02:47:03.749071 1085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790304178325258-13942-0/minicluster-data/ts-1/wal/instance:
uuid: "f74b2f820a3f4588990db88d7a18c341"
format_stamp: "Formatted at 2026-09-25 02:47:03 on dist-test-slave-kznw"
server_key: "02a122a27dbd5c79b6866018383e3fa6"
server_key_iv: "5c7a71bc94df66be8d0e5d3eb62a89d4"
server_key_version: "encryptionkey@0"
I20260925 02:47:03.752924 1085 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260925 02:47:03.755458 1100 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:47:03.756878 1085 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 02:47:03.756992 1085 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790304178325258-13942-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790304178325258-13942-0/minicluster-data/ts-1/wal
uuid: "f74b2f820a3f4588990db88d7a18c341"
format_stamp: "Formatted at 2026-09-25 02:47:03 on dist-test-slave-kznw"
server_key: "02a122a27dbd5c79b6866018383e3fa6"
server_key_iv: "5c7a71bc94df66be8d0e5d3eb62a89d4"
server_key_version: "encryptionkey@0"
I20260925 02:47:03.757097 1085 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790304178325258-13942-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790304178325258-13942-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790304178325258-13942-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:47:03.774245 1085 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:47:03.777405 1085 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:47:03.777593 1085 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:47:03.778193 1085 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:47:03.779098 1085 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:47:03.779157 1085 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:47:03.779214 1085 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:47:03.779242 1085 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:47:03.787631 1085 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.130:36555
I20260925 02:47:03.787662 1213 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.130:36555 every 8 connection(s)
I20260925 02:47:03.788507 1085 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790304178325258-13942-0/minicluster-data/ts-1/data/info.pb
I20260925 02:47:03.795329 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 1085
I20260925 02:47:03.795445 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790304178325258-13942-0/minicluster-data/ts-1/wal/instance
I20260925 02:47:03.795729 13942 external_mini_cluster.cc:1428] Setting key 288b0888579776539cac4a321214158c
Sep 25 02:47:03 dist-test-slave-kznw krb5kdc[844](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304423, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.130@KRBTEST.COM for kudu/127.13.157.190@KRBTEST.COM
I20260925 02:47:03.802313 928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:03.790344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.130:55613 (local address 127.13.157.190:44173)
0925 02:47:03.790489 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:03.790494 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:03.791388 (+ 894us) server_negotiation.cc:407] Connection header received
0925 02:47:03.792717 (+ 1329us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:03.792720 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:03.792769 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:03.792844 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:47:03.793932 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:03.794448 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:03.795662 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:03.795914 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:03.796005 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:03.798719 (+ 2714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:47:03.798722 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:47:03.798725 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:47:03.798781 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:47:03.800242 (+ 1461us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:47:03.800814 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:47:03.800818 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:47:03.800819 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:47:03.800902 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:47:03.801228 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:47:03.801230 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:47:03.801231 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:47:03.801392 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:47:03.801463 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:03.802082 (+ 619us) server_negotiation.cc:299] Negotiation successful
0925 02:47:03.802188 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260925 02:47:03.803483 1216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:03.790756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:44173 (local address 127.13.157.130:55613)
0925 02:47:03.791236 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0925 02:47:03.791279 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0925 02:47:03.792495 (+ 1216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:47:03.792988 (+ 493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:47:03.792995 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:47:03.793353 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:47:03.793820 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:47:03.793833 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:47:03.794533 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:47:03.794535 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:47:03.795537 (+ 1002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:47:03.795549 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:47:03.796080 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:47:03.796086 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:47:03.796272 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:03.796886 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:47:03.796907 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:47:03.798524 (+ 1617us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:47:03.800349 (+ 1825us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:47:03.800355 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:47:03.800372 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:47:03.800703 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:47:03.800981 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:47:03.800984 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:47:03.800986 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:47:03.801149 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:47:03.801479 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:47:03.801486 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:47:03.801814 (+ 328us) client_negotiation.cc:769] Sending connection context
0925 02:47:03.802031 (+ 217us) client_negotiation.cc:240] Negotiation successful
0925 02:47:03.802375 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":140,"threads_started":1}
I20260925 02:47:03.804617 1214 heartbeater.cc:347] Connected to a master server at 127.13.157.190:44173
I20260925 02:47:03.804873 1214 heartbeater.cc:464] Registering TS with master...
I20260925 02:47:03.805456 1214 heartbeater.cc:511] Master 127.13.157.190:44173 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.13.157.131@KRBTEST.COM; defaulting to no policy
I20260925 02:47:03.806574 889 ts_manager.cc:194] Registered new tserver with Master: f74b2f820a3f4588990db88d7a18c341 (127.13.157.130:36555)
I20260925 02:47:03.807160 889 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.157.130@KRBTEST.COM'} at 127.13.157.130:55613
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.157.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.157.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.131.keytab.
Entry for principal kudu/127.13.157.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.131.keytab.
I20260925 02:47:03.823948 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790304178325258-13942-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790304178325258-13942-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790304178325258-13942-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790304178325258-13942-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.157.131:0
--local_ip_for_outbound_sockets=127.13.157.131
--webserver_interface=127.13.157.131
--webserver_port=0
--tserver_master_addrs=127.13.157.190:44173
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.131.keytab
--principal=kudu/127.13.157.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:34181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:47:03.932894 1219 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:47:03.933156 1219 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:47:03.933198 1219 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:47:03.937350 1219 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:47:03.937407 1219 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:47:03.937469 1219 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.157.131
I20260925 02:47:03.942768 1219 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:34181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790304178325258-13942-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790304178325258-13942-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.131.keytab
--principal=kudu/127.13.157.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.157.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790304178325258-13942-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.157.131
--webserver_port=0
--tserver_master_addrs=127.13.157.190:44173
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1219
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.157.131
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790304178325258-13942-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:47:03.943795 1219 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:47:03.944764 1219 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:47:03.954550 1224 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:47:03.954591 1225 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:47:03.954685 1219 server_base.cc:1034] running on GCE node
W20260925 02:47:03.954555 1227 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:47:03.955279 1219 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:47:03.956048 1219 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:47:03.957211 1219 hybrid_clock.cc:648] HybridClock initialized: now 1790304423957190 us; error 32 us; skew 500 ppm
Sep 25 02:47:03 dist-test-slave-kznw krb5kdc[844](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304423, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:47:03.960085 1219 init.cc:377] Logged in from keytab as kudu/127.13.157.131@KRBTEST.COM (short username kudu)
I20260925 02:47:03.960959 1219 webserver.cc:460] Webserver started at http://127.13.157.131:37899/ using document root <none> and password file <none>
I20260925 02:47:03.961514 1219 fs_manager.cc:359] Metadata directory not provided
I20260925 02:47:03.961575 1219 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:47:03.961740 1219 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:47:03.963752 1219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790304178325258-13942-0/minicluster-data/ts-2/data/instance:
uuid: "5b0e187f6c4b40f398f310a5873bb282"
format_stamp: "Formatted at 2026-09-25 02:47:03 on dist-test-slave-kznw"
server_key: "21077b37e75c13b10b9d4591c7cac59f"
server_key_iv: "f2a616e7e1c315942babe4a081d7e765"
server_key_version: "encryptionkey@0"
I20260925 02:47:03.964274 1219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790304178325258-13942-0/minicluster-data/ts-2/wal/instance:
uuid: "5b0e187f6c4b40f398f310a5873bb282"
format_stamp: "Formatted at 2026-09-25 02:47:03 on dist-test-slave-kznw"
server_key: "21077b37e75c13b10b9d4591c7cac59f"
server_key_iv: "f2a616e7e1c315942babe4a081d7e765"
server_key_version: "encryptionkey@0"
I20260925 02:47:03.967912 1219 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260925 02:47:03.970188 1234 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:47:03.971427 1219 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260925 02:47:03.971529 1219 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790304178325258-13942-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790304178325258-13942-0/minicluster-data/ts-2/wal
uuid: "5b0e187f6c4b40f398f310a5873bb282"
format_stamp: "Formatted at 2026-09-25 02:47:03 on dist-test-slave-kznw"
server_key: "21077b37e75c13b10b9d4591c7cac59f"
server_key_iv: "f2a616e7e1c315942babe4a081d7e765"
server_key_version: "encryptionkey@0"
I20260925 02:47:03.971625 1219 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790304178325258-13942-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790304178325258-13942-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790304178325258-13942-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:47:03.984787 1219 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:47:03.987671 1219 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:47:03.987833 1219 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:47:03.988394 1219 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:47:03.989190 1219 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:47:03.989238 1219 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:47:03.989287 1219 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:47:03.989316 1219 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:47:03.997867 1219 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.131:40939
I20260925 02:47:03.997901 1347 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.131:40939 every 8 connection(s)
I20260925 02:47:03.998729 1219 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790304178325258-13942-0/minicluster-data/ts-2/data/info.pb
I20260925 02:47:03.999727 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 1219
I20260925 02:47:03.999841 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790304178325258-13942-0/minicluster-data/ts-2/wal/instance
I20260925 02:47:04.000174 13942 external_mini_cluster.cc:1428] Setting key 0b2d511dcd76399b21b76fbbede0efb5
Sep 25 02:47:04 dist-test-slave-kznw krb5kdc[844](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304423, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.131@KRBTEST.COM for kudu/127.13.157.190@KRBTEST.COM
I20260925 02:47:04.012511 928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:04.000613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.131:46045 (local address 127.13.157.190:44173)
0925 02:47:04.000754 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:04.000757 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:04.001721 (+ 964us) server_negotiation.cc:407] Connection header received
0925 02:47:04.003209 (+ 1488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:04.003212 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:04.003261 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:04.003325 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:47:04.004377 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:04.004893 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:04.005941 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:04.006152 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:04.006224 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:04.008887 (+ 2663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:47:04.008891 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:47:04.008893 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:47:04.008921 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:47:04.010266 (+ 1345us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:47:04.010849 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:47:04.010852 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:47:04.010854 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:47:04.010968 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:47:04.011289 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:47:04.011292 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:47:04.011294 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:47:04.011493 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:47:04.011578 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:04.012187 (+ 609us) server_negotiation.cc:299] Negotiation successful
0925 02:47:04.012318 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260925 02:47:04.013515 1350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:04.001069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:44173 (local address 127.13.157.131:46045)
0925 02:47:04.001568 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0925 02:47:04.001626 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0925 02:47:04.003012 (+ 1386us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:47:04.003454 (+ 442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:47:04.003461 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:47:04.003791 (+ 330us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:47:04.004268 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:47:04.004281 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:47:04.004977 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:47:04.004979 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:47:04.005850 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:47:04.005858 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:47:04.006271 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:47:04.006273 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:47:04.006410 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:04.006977 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:47:04.006994 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:47:04.008713 (+ 1719us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:47:04.010393 (+ 1680us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:47:04.010399 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:47:04.010414 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:47:04.010757 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:47:04.011056 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:47:04.011059 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:47:04.011061 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:47:04.011203 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:47:04.011629 (+ 426us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:47:04.011634 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:47:04.011921 (+ 287us) client_negotiation.cc:769] Sending connection context
0925 02:47:04.012140 (+ 219us) client_negotiation.cc:240] Negotiation successful
0925 02:47:04.012452 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":115,"threads_started":1}
I20260925 02:47:04.014598 1348 heartbeater.cc:347] Connected to a master server at 127.13.157.190:44173
I20260925 02:47:04.014793 1348 heartbeater.cc:464] Registering TS with master...
I20260925 02:47:04.015260 1348 heartbeater.cc:511] Master 127.13.157.190:44173 requested a full tablet report, sending...
I20260925 02:47:04.016216 888 ts_manager.cc:194] Registered new tserver with Master: 5b0e187f6c4b40f398f310a5873bb282 (127.13.157.131:40939)
I20260925 02:47:04.016791 888 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.157.131@KRBTEST.COM'} at 127.13.157.131:46045
I20260925 02:47:04.026335 13942 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 02:47:04.034937 928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:04.028301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43516 (local address 127.13.157.190:44173)
0925 02:47:04.028442 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:04.028447 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:04.028579 (+ 132us) server_negotiation.cc:407] Connection header received
0925 02:47:04.028680 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:04.028682 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:04.028722 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:04.028781 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:47:04.029361 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:04.029858 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:04.030667 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:04.030877 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:04.030975 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:04.032291 (+ 1316us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:47:04.032293 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:47:04.032295 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:47:04.032317 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:47:04.033550 (+ 1233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:47:04.033933 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:47:04.033935 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:47:04.033937 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:47:04.034000 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:47:04.034182 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:47:04.034184 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:47:04.034185 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:47:04.034371 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:47:04.034467 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:04.034670 (+ 203us) server_negotiation.cc:299] Negotiation successful
0925 02:47:04.034759 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260925 02:47:04.037626 888 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43516:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 02:47:04.040716 888 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 02:47:04.055303 1364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:04.050491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34682 (local address 127.13.157.130:36555)
0925 02:47:04.050841 (+ 350us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:04.050854 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:04.050874 (+ 20us) server_negotiation.cc:407] Connection header received
0925 02:47:04.050963 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:04.050968 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:04.051172 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:04.051342 (+ 170us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:47:04.051518 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:04.052577 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:04.054016 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:04.054580 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:04.054686 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:04.054820 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:04.055045 (+ 225us) server_negotiation.cc:299] Negotiation successful
0925 02:47:04.055145 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":255,"thread_start_us":96,"threads_started":1}
I20260925 02:47:04.055292 1359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:04.049439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.130:36555 (local address 127.0.0.1:34682)
0925 02:47:04.049839 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0925 02:47:04.049864 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0925 02:47:04.050043 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:47:04.051319 (+ 1276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:47:04.051322 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:47:04.051338 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:47:04.051427 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:47:04.051433 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:47:04.052701 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:47:04.052705 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:47:04.053920 (+ 1215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:47:04.053927 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:47:04.054686 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:47:04.054689 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:47:04.054834 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:04.054877 (+ 43us) client_negotiation.cc:769] Sending connection context
0925 02:47:04.055015 (+ 138us) client_negotiation.cc:240] Negotiation successful
0925 02:47:04.055157 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":352,"mutex_wait_us":74,"thread_start_us":103,"threads_started":1}
I20260925 02:47:04.055315 1362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:04.049910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.131:40939 (local address 127.0.0.1:50142)
0925 02:47:04.050535 (+ 625us) negotiation.cc:107] Waiting for socket to connect
0925 02:47:04.050548 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 02:47:04.050622 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:47:04.051160 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:47:04.051163 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:47:04.051182 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:47:04.051270 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:47:04.051277 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:47:04.052302 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:47:04.052305 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:47:04.053704 (+ 1399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:47:04.053714 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:47:04.054791 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:47:04.054795 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:47:04.054949 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:04.054973 (+ 24us) client_negotiation.cc:769] Sending connection context
0925 02:47:04.055033 (+ 60us) client_negotiation.cc:240] Negotiation successful
0925 02:47:04.055163 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":563,"mutex_wait_us":133,"thread_start_us":111,"threads_started":1}
I20260925 02:47:04.056007 1363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:04.050069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50142 (local address 127.13.157.131:40939)
0925 02:47:04.050745 (+ 676us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:04.050756 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:04.050779 (+ 23us) server_negotiation.cc:407] Connection header received
0925 02:47:04.050843 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:04.050847 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:04.051038 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:04.051173 (+ 135us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:47:04.051392 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:04.052179 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:04.053837 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:04.054655 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:04.055128 (+ 473us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:04.055245 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:04.055294 (+ 49us) server_negotiation.cc:299] Negotiation successful
0925 02:47:04.055861 (+ 567us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":575,"thread_start_us":135,"threads_started":1}
I20260925 02:47:04.056036 1361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:04.049096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.129:37019 (local address 127.0.0.1:34130)
0925 02:47:04.049813 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0925 02:47:04.049864 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0925 02:47:04.050034 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:47:04.050881 (+ 847us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:47:04.050887 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:47:04.050934 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:47:04.051064 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:47:04.051071 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:47:04.052528 (+ 1457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:47:04.052541 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:47:04.053701 (+ 1160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:47:04.053710 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:47:04.054987 (+ 1277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:47:04.054990 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:47:04.055071 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:04.055742 (+ 671us) client_negotiation.cc:769] Sending connection context
0925 02:47:04.055805 (+ 63us) client_negotiation.cc:240] Negotiation successful
0925 02:47:04.055924 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":538,"mutex_wait_us":305,"thread_start_us":95,"threads_started":1}
I20260925 02:47:04.056106 1360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:04.049240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34130 (local address 127.13.157.129:37019)
0925 02:47:04.049698 (+ 458us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:04.049710 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:04.049963 (+ 253us) server_negotiation.cc:407] Connection header received
0925 02:47:04.050218 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:04.050224 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:04.050433 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:04.050604 (+ 171us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:47:04.051196 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:04.052376 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:04.053818 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:04.054655 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:04.054776 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:04.054928 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:04.055850 (+ 922us) server_negotiation.cc:299] Negotiation successful
0925 02:47:04.055946 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":147,"threads_started":1}
I20260925 02:47:04.058275 1148 tablet_service.cc:1467] Processing CreateTablet for tablet fab5560f38a7476d9341ccc744d3bb69 (DEFAULT_TABLE table=test-table [id=cb656f53e96e469fb12c15e91db651ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:47:04.059319 1148 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fab5560f38a7476d9341ccc744d3bb69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:47:04.058985 1282 tablet_service.cc:1467] Processing CreateTablet for tablet fab5560f38a7476d9341ccc744d3bb69 (DEFAULT_TABLE table=test-table [id=cb656f53e96e469fb12c15e91db651ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:47:04.058986 1014 tablet_service.cc:1467] Processing CreateTablet for tablet fab5560f38a7476d9341ccc744d3bb69 (DEFAULT_TABLE table=test-table [id=cb656f53e96e469fb12c15e91db651ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:47:04.060012 1282 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fab5560f38a7476d9341ccc744d3bb69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:47:04.060012 1014 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fab5560f38a7476d9341ccc744d3bb69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:47:04.065883 1365 tablet_bootstrap.cc:492] T fab5560f38a7476d9341ccc744d3bb69 P f74b2f820a3f4588990db88d7a18c341: Bootstrap starting.
I20260925 02:47:04.067071 1366 tablet_bootstrap.cc:492] T fab5560f38a7476d9341ccc744d3bb69 P 3112b67b7cf0405a93b31605ae5e7800: Bootstrap starting.
I20260925 02:47:04.067478 1367 tablet_bootstrap.cc:492] T fab5560f38a7476d9341ccc744d3bb69 P 5b0e187f6c4b40f398f310a5873bb282: Bootstrap starting.
I20260925 02:47:04.068496 1365 tablet_bootstrap.cc:654] T fab5560f38a7476d9341ccc744d3bb69 P f74b2f820a3f4588990db88d7a18c341: Neither blocks nor log segments found. Creating new log.
I20260925 02:47:04.069800 1365 log.cc:824] T fab5560f38a7476d9341ccc744d3bb69 P f74b2f820a3f4588990db88d7a18c341: Log is configured to *not* fsync() on all Append() calls
I20260925 02:47:04.071103 1366 tablet_bootstrap.cc:654] T fab5560f38a7476d9341ccc744d3bb69 P 3112b67b7cf0405a93b31605ae5e7800: Neither blocks nor log segments found. Creating new log.
I20260925 02:47:04.071101 1367 tablet_bootstrap.cc:654] T fab5560f38a7476d9341ccc744d3bb69 P 5b0e187f6c4b40f398f310a5873bb282: Neither blocks nor log segments found. Creating new log.
I20260925 02:47:04.071818 1365 tablet_bootstrap.cc:492] T fab5560f38a7476d9341ccc744d3bb69 P f74b2f820a3f4588990db88d7a18c341: No bootstrap required, opened a new log
I20260925 02:47:04.071996 1365 ts_tablet_manager.cc:1397] T fab5560f38a7476d9341ccc744d3bb69 P f74b2f820a3f4588990db88d7a18c341: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.004s
I20260925 02:47:04.072345 1366 log.cc:824] T fab5560f38a7476d9341ccc744d3bb69 P 3112b67b7cf0405a93b31605ae5e7800: Log is configured to *not* fsync() on all Append() calls
I20260925 02:47:04.072345 1367 log.cc:824] T fab5560f38a7476d9341ccc744d3bb69 P 5b0e187f6c4b40f398f310a5873bb282: Log is configured to *not* fsync() on all Append() calls
I20260925 02:47:04.073998 1367 tablet_bootstrap.cc:492] T fab5560f38a7476d9341ccc744d3bb69 P 5b0e187f6c4b40f398f310a5873bb282: No bootstrap required, opened a new log
I20260925 02:47:04.074174 1367 ts_tablet_manager.cc:1397] T fab5560f38a7476d9341ccc744d3bb69 P 5b0e187f6c4b40f398f310a5873bb282: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260925 02:47:04.074256 1366 tablet_bootstrap.cc:492] T fab5560f38a7476d9341ccc744d3bb69 P 3112b67b7cf0405a93b31605ae5e7800: No bootstrap required, opened a new log
I20260925 02:47:04.074432 1366 ts_tablet_manager.cc:1397] T fab5560f38a7476d9341ccc744d3bb69 P 3112b67b7cf0405a93b31605ae5e7800: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20260925 02:47:04.075615 1365 raft_consensus.cc:348] T fab5560f38a7476d9341ccc744d3bb69 P f74b2f820a3f4588990db88d7a18c341 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3112b67b7cf0405a93b31605ae5e7800" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 37019 } } peers { permanent_uuid: "5b0e187f6c4b40f398f310a5873bb282" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 40939 } } peers { permanent_uuid: "f74b2f820a3f4588990db88d7a18c341" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 36555 } }
I20260925 02:47:04.075886 1365 raft_consensus.cc:374] T fab5560f38a7476d9341ccc744d3bb69 P f74b2f820a3f4588990db88d7a18c341 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:47:04.075937 1365 raft_consensus.cc:729] T fab5560f38a7476d9341ccc744d3bb69 P f74b2f820a3f4588990db88d7a18c341 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f74b2f820a3f4588990db88d7a18c341, State: Initialized, Role: FOLLOWER
I20260925 02:47:04.076390 1365 consensus_queue.cc:260] T fab5560f38a7476d9341ccc744d3bb69 P f74b2f820a3f4588990db88d7a18c341 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3112b67b7cf0405a93b31605ae5e7800" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 37019 } } peers { permanent_uuid: "5b0e187f6c4b40f398f310a5873bb282" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 40939 } } peers { permanent_uuid: "f74b2f820a3f4588990db88d7a18c341" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 36555 } }
I20260925 02:47:04.077093 1214 heartbeater.cc:503] Master 127.13.157.190:44173 was elected leader, sending a full tablet report...
I20260925 02:47:04.077445 1365 ts_tablet_manager.cc:1428] T fab5560f38a7476d9341ccc744d3bb69 P f74b2f820a3f4588990db88d7a18c341: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260925 02:47:04.077448 1367 raft_consensus.cc:348] T fab5560f38a7476d9341ccc744d3bb69 P 5b0e187f6c4b40f398f310a5873bb282 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3112b67b7cf0405a93b31605ae5e7800" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 37019 } } peers { permanent_uuid: "5b0e187f6c4b40f398f310a5873bb282" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 40939 } } peers { permanent_uuid: "f74b2f820a3f4588990db88d7a18c341" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 36555 } }
I20260925 02:47:04.077689 1367 raft_consensus.cc:374] T fab5560f38a7476d9341ccc744d3bb69 P 5b0e187f6c4b40f398f310a5873bb282 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:47:04.077737 1367 raft_consensus.cc:729] T fab5560f38a7476d9341ccc744d3bb69 P 5b0e187f6c4b40f398f310a5873bb282 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b0e187f6c4b40f398f310a5873bb282, State: Initialized, Role: FOLLOWER
I20260925 02:47:04.078159 1367 consensus_queue.cc:260] T fab5560f38a7476d9341ccc744d3bb69 P 5b0e187f6c4b40f398f310a5873bb282 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3112b67b7cf0405a93b31605ae5e7800" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 37019 } } peers { permanent_uuid: "5b0e187f6c4b40f398f310a5873bb282" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 40939 } } peers { permanent_uuid: "f74b2f820a3f4588990db88d7a18c341" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 36555 } }
I20260925 02:47:04.078931 1348 heartbeater.cc:503] Master 127.13.157.190:44173 was elected leader, sending a full tablet report...
I20260925 02:47:04.078867 1366 raft_consensus.cc:348] T fab5560f38a7476d9341ccc744d3bb69 P 3112b67b7cf0405a93b31605ae5e7800 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3112b67b7cf0405a93b31605ae5e7800" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 37019 } } peers { permanent_uuid: "5b0e187f6c4b40f398f310a5873bb282" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 40939 } } peers { permanent_uuid: "f74b2f820a3f4588990db88d7a18c341" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 36555 } }
I20260925 02:47:04.079162 1366 raft_consensus.cc:374] T fab5560f38a7476d9341ccc744d3bb69 P 3112b67b7cf0405a93b31605ae5e7800 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:47:04.079205 1367 ts_tablet_manager.cc:1428] T fab5560f38a7476d9341ccc744d3bb69 P 5b0e187f6c4b40f398f310a5873bb282: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260925 02:47:04.079213 1366 raft_consensus.cc:729] T fab5560f38a7476d9341ccc744d3bb69 P 3112b67b7cf0405a93b31605ae5e7800 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3112b67b7cf0405a93b31605ae5e7800, State: Initialized, Role: FOLLOWER
I20260925 02:47:04.079625 1366 consensus_queue.cc:260] T fab5560f38a7476d9341ccc744d3bb69 P 3112b67b7cf0405a93b31605ae5e7800 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3112b67b7cf0405a93b31605ae5e7800" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 37019 } } peers { permanent_uuid: "5b0e187f6c4b40f398f310a5873bb282" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 40939 } } peers { permanent_uuid: "f74b2f820a3f4588990db88d7a18c341" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 36555 } }
I20260925 02:47:04.080351 1080 heartbeater.cc:503] Master 127.13.157.190:44173 was elected leader, sending a full tablet report...
I20260925 02:47:04.080571 1366 ts_tablet_manager.cc:1428] T fab5560f38a7476d9341ccc744d3bb69 P 3112b67b7cf0405a93b31605ae5e7800: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260925 02:47:04.140901 1371 raft_consensus.cc:482] T fab5560f38a7476d9341ccc744d3bb69 P f74b2f820a3f4588990db88d7a18c341 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 02:47:04.141096 1371 raft_consensus.cc:504] T fab5560f38a7476d9341ccc744d3bb69 P f74b2f820a3f4588990db88d7a18c341 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3112b67b7cf0405a93b31605ae5e7800" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 37019 } } peers { permanent_uuid: "5b0e187f6c4b40f398f310a5873bb282" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 40939 } } peers { permanent_uuid: "f74b2f820a3f4588990db88d7a18c341" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 36555 } }
I20260925 02:47:04.142191 1371 leader_election.cc:288] T fab5560f38a7476d9341ccc744d3bb69 P f74b2f820a3f4588990db88d7a18c341 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3112b67b7cf0405a93b31605ae5e7800 (127.13.157.129:37019), 5b0e187f6c4b40f398f310a5873bb282 (127.13.157.131:40939)
I20260925 02:47:04.146975 1360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:04.142567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.130:56303 (local address 127.13.157.129:37019)
0925 02:47:04.142695 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:04.142701 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:04.142720 (+ 19us) server_negotiation.cc:407] Connection header received
0925 02:47:04.142783 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:04.142786 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:04.142836 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:04.142944 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:47:04.143306 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:04.144083 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:04.145647 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:04.146285 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:04.146394 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:04.146447 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:04.146747 (+ 300us) server_negotiation.cc:299] Negotiation successful
0925 02:47:04.146819 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260925 02:47:04.146974 1363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:04.142856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.130:37151 (local address 127.13.157.131:40939)
0925 02:47:04.143013 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:04.143019 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:04.143129 (+ 110us) server_negotiation.cc:407] Connection header received
0925 02:47:04.143232 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:04.143234 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:04.143280 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:04.143358 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:47:04.143565 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:04.144255 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:04.145645 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:04.146272 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:04.146394 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:04.146447 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:04.146743 (+ 296us) server_negotiation.cc:299] Negotiation successful
0925 02:47:04.146819 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260925 02:47:04.147027 1216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:04.142444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.129:37019 (local address 127.13.157.130:56303)
0925 02:47:04.142559 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0925 02:47:04.142576 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 02:47:04.142675 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:47:04.143064 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:47:04.143067 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:47:04.143088 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:47:04.143207 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:47:04.143214 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:47:04.144206 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:47:04.144209 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:47:04.145493 (+ 1284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:47:04.145505 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:47:04.146392 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:47:04.146396 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:47:04.146646 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:04.146663 (+ 17us) client_negotiation.cc:769] Sending connection context
0925 02:47:04.146722 (+ 59us) client_negotiation.cc:240] Negotiation successful
0925 02:47:04.146853 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30,"mutex_wait_us":100}
I20260925 02:47:04.147246 1374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:04.142735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.131:40939 (local address 127.13.157.130:37151)
0925 02:47:04.143065 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0925 02:47:04.143077 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 02:47:04.143157 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:47:04.143365 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:47:04.143367 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:47:04.143381 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:47:04.143475 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:47:04.143480 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:47:04.144358 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:47:04.144362 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:47:04.145525 (+ 1163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:47:04.145532 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:47:04.146386 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:47:04.146390 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:47:04.146592 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:04.146641 (+ 49us) client_negotiation.cc:769] Sending connection context
0925 02:47:04.146721 (+ 80us) client_negotiation.cc:240] Negotiation successful
0925 02:47:04.146846 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"mutex_wait_us":102,"thread_start_us":158,"threads_started":1}
I20260925 02:47:04.147719 1034 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fab5560f38a7476d9341ccc744d3bb69" candidate_uuid: "f74b2f820a3f4588990db88d7a18c341" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3112b67b7cf0405a93b31605ae5e7800" is_pre_election: true
I20260925 02:47:04.147763 1302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fab5560f38a7476d9341ccc744d3bb69" candidate_uuid: "f74b2f820a3f4588990db88d7a18c341" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b0e187f6c4b40f398f310a5873bb282" is_pre_election: true
I20260925 02:47:04.148067 1302 raft_consensus.cc:2454] T fab5560f38a7476d9341ccc744d3bb69 P 5b0e187f6c4b40f398f310a5873bb282 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f74b2f820a3f4588990db88d7a18c341 in term 0.
I20260925 02:47:04.148068 1034 raft_consensus.cc:2454] T fab5560f38a7476d9341ccc744d3bb69 P 3112b67b7cf0405a93b31605ae5e7800 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f74b2f820a3f4588990db88d7a18c341 in term 0.
I20260925 02:47:04.148553 1103 leader_election.cc:302] T fab5560f38a7476d9341ccc744d3bb69 P f74b2f820a3f4588990db88d7a18c341 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3112b67b7cf0405a93b31605ae5e7800, f74b2f820a3f4588990db88d7a18c341; no voters:
I20260925 02:47:04.148809 1371 raft_consensus.cc:2781] T fab5560f38a7476d9341ccc744d3bb69 P f74b2f820a3f4588990db88d7a18c341 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 02:47:04.148897 1371 raft_consensus.cc:482] T fab5560f38a7476d9341ccc744d3bb69 P f74b2f820a3f4588990db88d7a18c341 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 02:47:04.148929 1371 raft_consensus.cc:3037] T fab5560f38a7476d9341ccc744d3bb69 P f74b2f820a3f4588990db88d7a18c341 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:47:04.149977 1371 raft_consensus.cc:504] T fab5560f38a7476d9341ccc744d3bb69 P f74b2f820a3f4588990db88d7a18c341 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3112b67b7cf0405a93b31605ae5e7800" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 37019 } } peers { permanent_uuid: "5b0e187f6c4b40f398f310a5873bb282" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 40939 } } peers { permanent_uuid: "f74b2f820a3f4588990db88d7a18c341" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 36555 } }
I20260925 02:47:04.150301 1371 leader_election.cc:288] T fab5560f38a7476d9341ccc744d3bb69 P f74b2f820a3f4588990db88d7a18c341 [CANDIDATE]: Term 1 election: Requested vote from peers 3112b67b7cf0405a93b31605ae5e7800 (127.13.157.129:37019), 5b0e187f6c4b40f398f310a5873bb282 (127.13.157.131:40939)
I20260925 02:47:04.150743 1034 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fab5560f38a7476d9341ccc744d3bb69" candidate_uuid: "f74b2f820a3f4588990db88d7a18c341" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3112b67b7cf0405a93b31605ae5e7800"
I20260925 02:47:04.150871 1034 raft_consensus.cc:3037] T fab5560f38a7476d9341ccc744d3bb69 P 3112b67b7cf0405a93b31605ae5e7800 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:47:04.151044 1302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fab5560f38a7476d9341ccc744d3bb69" candidate_uuid: "f74b2f820a3f4588990db88d7a18c341" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b0e187f6c4b40f398f310a5873bb282"
I20260925 02:47:04.151172 1302 raft_consensus.cc:3037] T fab5560f38a7476d9341ccc744d3bb69 P 5b0e187f6c4b40f398f310a5873bb282 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:47:04.151952 1034 raft_consensus.cc:2454] T fab5560f38a7476d9341ccc744d3bb69 P 3112b67b7cf0405a93b31605ae5e7800 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f74b2f820a3f4588990db88d7a18c341 in term 1.
I20260925 02:47:04.152235 1103 leader_election.cc:302] T fab5560f38a7476d9341ccc744d3bb69 P f74b2f820a3f4588990db88d7a18c341 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3112b67b7cf0405a93b31605ae5e7800, f74b2f820a3f4588990db88d7a18c341; no voters:
I20260925 02:47:04.152276 1302 raft_consensus.cc:2454] T fab5560f38a7476d9341ccc744d3bb69 P 5b0e187f6c4b40f398f310a5873bb282 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f74b2f820a3f4588990db88d7a18c341 in term 1.
I20260925 02:47:04.152408 1371 raft_consensus.cc:2781] T fab5560f38a7476d9341ccc744d3bb69 P f74b2f820a3f4588990db88d7a18c341 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:47:04.152639 1371 raft_consensus.cc:686] T fab5560f38a7476d9341ccc744d3bb69 P f74b2f820a3f4588990db88d7a18c341 [term 1 LEADER]: Becoming Leader. State: Replica: f74b2f820a3f4588990db88d7a18c341, State: Running, Role: LEADER
I20260925 02:47:04.152954 1371 consensus_queue.cc:237] T fab5560f38a7476d9341ccc744d3bb69 P f74b2f820a3f4588990db88d7a18c341 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3112b67b7cf0405a93b31605ae5e7800" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 37019 } } peers { permanent_uuid: "5b0e187f6c4b40f398f310a5873bb282" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 40939 } } peers { permanent_uuid: "f74b2f820a3f4588990db88d7a18c341" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 36555 } }
I20260925 02:47:04.156257 888 catalog_manager.cc:5495] T fab5560f38a7476d9341ccc744d3bb69 P f74b2f820a3f4588990db88d7a18c341 reported cstate change: term changed from 0 to 1, leader changed from <none> to f74b2f820a3f4588990db88d7a18c341 (127.13.157.130). New cstate: current_term: 1 leader_uuid: "f74b2f820a3f4588990db88d7a18c341" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3112b67b7cf0405a93b31605ae5e7800" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 37019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b0e187f6c4b40f398f310a5873bb282" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 40939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f74b2f820a3f4588990db88d7a18c341" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 36555 } health_report { overall_health: HEALTHY } } }
I20260925 02:47:04.176047 1364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:04.172607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34688 (local address 127.13.157.130:36555)
0925 02:47:04.172731 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:04.172735 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:04.172748 (+ 13us) server_negotiation.cc:407] Connection header received
0925 02:47:04.172828 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:04.172830 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:04.172888 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:04.172976 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 02:47:04.173242 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:04.173738 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:04.174564 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:04.174822 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:04.174935 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:04.175234 (+ 299us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 02:47:04.175646 (+ 412us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 02:47:04.175750 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:04.175855 (+ 105us) server_negotiation.cc:299] Negotiation successful
0925 02:47:04.175911 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260925 02:47:04.181769 1034 raft_consensus.cc:1261] T fab5560f38a7476d9341ccc744d3bb69 P 3112b67b7cf0405a93b31605ae5e7800 [term 1 FOLLOWER]: Refusing update from remote peer f74b2f820a3f4588990db88d7a18c341: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:47:04.181769 1302 raft_consensus.cc:1261] T fab5560f38a7476d9341ccc744d3bb69 P 5b0e187f6c4b40f398f310a5873bb282 [term 1 FOLLOWER]: Refusing update from remote peer f74b2f820a3f4588990db88d7a18c341: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:47:04.182466 1371 consensus_queue.cc:1035] T fab5560f38a7476d9341ccc744d3bb69 P f74b2f820a3f4588990db88d7a18c341 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3112b67b7cf0405a93b31605ae5e7800" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 37019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:47:04.182667 1375 consensus_queue.cc:1035] T fab5560f38a7476d9341ccc744d3bb69 P f74b2f820a3f4588990db88d7a18c341 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b0e187f6c4b40f398f310a5873bb282" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 40939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:47:04.191028 1378 mvcc.cc:205] Tried to move back new op lower bound from 7333086921441181696 to 7333086921333010432. Current Snapshot: MvccSnapshot[applied={T|T < 7333086921441181696}]
I20260925 02:47:04.193138 1379 mvcc.cc:205] Tried to move back new op lower bound from 7333086921441181696 to 7333086921333010432. Current Snapshot: MvccSnapshot[applied={T|T < 7333086921441181696 or (T in {7333086921441181696})}]
I20260925 02:47:04.200560 888 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43516:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 02:47:04.200748 888 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43516:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 02:47:04.203195 888 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6419840f89ab45d5ad6d9350ad4e4220: Sending DeleteTablet for 3 replicas of tablet fab5560f38a7476d9341ccc744d3bb69
I20260925 02:47:04.204017 1148 tablet_service.cc:1514] Processing DeleteTablet for tablet fab5560f38a7476d9341ccc744d3bb69 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:47:04 UTC) from {username='kudu', principal='kudu/127.13.157.190@KRBTEST.COM'} at 127.0.0.1:34682
I20260925 02:47:04.204025 1014 tablet_service.cc:1514] Processing DeleteTablet for tablet fab5560f38a7476d9341ccc744d3bb69 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:47:04 UTC) from {username='kudu', principal='kudu/127.13.157.190@KRBTEST.COM'} at 127.0.0.1:34130
I20260925 02:47:04.204167 1282 tablet_service.cc:1514] Processing DeleteTablet for tablet fab5560f38a7476d9341ccc744d3bb69 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:47:04 UTC) from {username='kudu', principal='kudu/127.13.157.190@KRBTEST.COM'} at 127.0.0.1:50142
I20260925 02:47:04.204605 1388 tablet_replica.cc:330] T fab5560f38a7476d9341ccc744d3bb69 P 3112b67b7cf0405a93b31605ae5e7800: stopping tablet replica
I20260925 02:47:04.204792 1390 tablet_replica.cc:330] T fab5560f38a7476d9341ccc744d3bb69 P 5b0e187f6c4b40f398f310a5873bb282: stopping tablet replica
I20260925 02:47:04.204941 1388 raft_consensus.cc:2229] T fab5560f38a7476d9341ccc744d3bb69 P 3112b67b7cf0405a93b31605ae5e7800 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 02:47:04.205152 1390 raft_consensus.cc:2229] T fab5560f38a7476d9341ccc744d3bb69 P 5b0e187f6c4b40f398f310a5873bb282 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 02:47:04.205265 1388 raft_consensus.cc:2258] T fab5560f38a7476d9341ccc744d3bb69 P 3112b67b7cf0405a93b31605ae5e7800 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:47:04.205334 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 951
I20260925 02:47:04.206619 1389 tablet_replica.cc:330] T fab5560f38a7476d9341ccc744d3bb69 P f74b2f820a3f4588990db88d7a18c341: stopping tablet replica
I20260925 02:47:04.206746 1390 raft_consensus.cc:2258] T fab5560f38a7476d9341ccc744d3bb69 P 5b0e187f6c4b40f398f310a5873bb282 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:47:04.206893 1389 raft_consensus.cc:2229] T fab5560f38a7476d9341ccc744d3bb69 P f74b2f820a3f4588990db88d7a18c341 [term 1 LEADER]: Raft consensus shutting down.
I20260925 02:47:04.207306 1389 raft_consensus.cc:2258] T fab5560f38a7476d9341ccc744d3bb69 P f74b2f820a3f4588990db88d7a18c341 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:47:04.208611 1389 ts_tablet_manager.cc:1905] T fab5560f38a7476d9341ccc744d3bb69 P f74b2f820a3f4588990db88d7a18c341: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:47:04.208611 1390 ts_tablet_manager.cc:1905] T fab5560f38a7476d9341ccc744d3bb69 P 5b0e187f6c4b40f398f310a5873bb282: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:47:04.211256 1390 ts_tablet_manager.cc:1918] T fab5560f38a7476d9341ccc744d3bb69 P 5b0e187f6c4b40f398f310a5873bb282: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 02:47:04.211256 1389 ts_tablet_manager.cc:1918] T fab5560f38a7476d9341ccc744d3bb69 P f74b2f820a3f4588990db88d7a18c341: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 02:47:04.211356 1389 log.cc:1196] T fab5560f38a7476d9341ccc744d3bb69 P f74b2f820a3f4588990db88d7a18c341: Deleting WAL directory at /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790304178325258-13942-0/minicluster-data/ts-1/wal/wals/fab5560f38a7476d9341ccc744d3bb69
I20260925 02:47:04.211356 1390 log.cc:1196] T fab5560f38a7476d9341ccc744d3bb69 P 5b0e187f6c4b40f398f310a5873bb282: Deleting WAL directory at /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790304178325258-13942-0/minicluster-data/ts-2/wal/wals/fab5560f38a7476d9341ccc744d3bb69
I20260925 02:47:04.211685 1390 ts_tablet_manager.cc:1939] T fab5560f38a7476d9341ccc744d3bb69 P 5b0e187f6c4b40f398f310a5873bb282: Deleting consensus metadata
I20260925 02:47:04.211685 1389 ts_tablet_manager.cc:1939] T fab5560f38a7476d9341ccc744d3bb69 P f74b2f820a3f4588990db88d7a18c341: Deleting consensus metadata
I20260925 02:47:04.212816 876 catalog_manager.cc:4841] TS f74b2f820a3f4588990db88d7a18c341 (127.13.157.130:36555): tablet fab5560f38a7476d9341ccc744d3bb69 (table test-table [id=cb656f53e96e469fb12c15e91db651ca]) successfully deleted
I20260925 02:47:04.212882 875 catalog_manager.cc:4841] TS 5b0e187f6c4b40f398f310a5873bb282 (127.13.157.131:40939): tablet fab5560f38a7476d9341ccc744d3bb69 (table test-table [id=cb656f53e96e469fb12c15e91db651ca]) successfully deleted
W20260925 02:47:04.216372 875 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.157.129:37019 (error 108)
I20260925 02:47:04.217166 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 1085
I20260925 02:47:04.217379 1361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:04.217084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.129:37019 (local address 127.0.0.1:34146)
0925 02:47:04.217221 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0925 02:47:04.217288 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.129:37019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260925 02:47:04.217929 875 catalog_manager.cc:4584] TS 3112b67b7cf0405a93b31605ae5e7800 (127.13.157.129:37019): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet fab5560f38a7476d9341ccc744d3bb69: Network error: Client connection negotiation failed: client connection to 127.13.157.129:37019: connect: Connection refused (error 111)
I20260925 02:47:04.222292 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 1219
I20260925 02:47:04.226943 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 857
2026-09-25T02:47:04Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2190 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-09-25T02:47:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T02:47:04Z Disabled control of system clock
I20260925 02:47:04.255329 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790304178325258-13942-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790304178325258-13942-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790304178325258-13942-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.157.190:36871
--webserver_interface=127.13.157.190
--webserver_port=0
--builtin_ntp_servers=127.13.157.148:43429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.157.190:36871
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260925 02:47:04.362318 1395 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:47:04.362602 1395 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:47:04.362636 1395 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:47:04.366565 1395 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:47:04.366623 1395 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:47:04.366637 1395 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:47:04.366653 1395 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:47:04.366667 1395 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:47:04.371332 1395 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:43429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790304178325258-13942-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790304178325258-13942-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.157.190:36871
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.157.190:36871
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.157.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1395
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790304178325258-13942-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:47:04.372351 1395 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:47:04.373697 1395 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 02:47:04.381852 1395 server_base.cc:1034] running on GCE node
W20260925 02:47:04.381791 1403 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:47:04.381790 1400 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:47:04.381819 1401 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:47:04.382580 1395 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:47:04.383651 1395 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:47:04.384799 1395 hybrid_clock.cc:648] HybridClock initialized: now 1790304424384776 us; error 32 us; skew 500 ppm
I20260925 02:47:04.386631 1395 webserver.cc:460] Webserver started at http://127.13.157.190:40397/ using document root <none> and password file <none>
I20260925 02:47:04.387226 1395 fs_manager.cc:359] Metadata directory not provided
I20260925 02:47:04.387286 1395 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:47:04.387481 1395 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:47:04.389456 1395 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790304178325258-13942-0/minicluster-data/master-0/data/instance:
uuid: "62fa485bead44f2ebb1a9f7e65c47d1a"
format_stamp: "Formatted at 2026-09-25 02:47:04 on dist-test-slave-kznw"
server_key: "9a76ae791956c1eeef6f5510481b5857"
server_key_iv: "6d989cb36f1ab357311bf45e35659d96"
server_key_version: "encryptionkey@0"
I20260925 02:47:04.389990 1395 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790304178325258-13942-0/minicluster-data/master-0/wal/instance:
uuid: "62fa485bead44f2ebb1a9f7e65c47d1a"
format_stamp: "Formatted at 2026-09-25 02:47:04 on dist-test-slave-kznw"
server_key: "9a76ae791956c1eeef6f5510481b5857"
server_key_iv: "6d989cb36f1ab357311bf45e35659d96"
server_key_version: "encryptionkey@0"
I20260925 02:47:04.393534 1395 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260925 02:47:04.395932 1409 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:47:04.397192 1395 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260925 02:47:04.397293 1395 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790304178325258-13942-0/minicluster-data/master-0/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790304178325258-13942-0/minicluster-data/master-0/wal
uuid: "62fa485bead44f2ebb1a9f7e65c47d1a"
format_stamp: "Formatted at 2026-09-25 02:47:04 on dist-test-slave-kznw"
server_key: "9a76ae791956c1eeef6f5510481b5857"
server_key_iv: "6d989cb36f1ab357311bf45e35659d96"
server_key_version: "encryptionkey@0"
I20260925 02:47:04.397400 1395 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790304178325258-13942-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790304178325258-13942-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790304178325258-13942-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:47:04.421066 1395 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:47:04.421768 1395 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:47:04.421942 1395 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:47:04.429287 1395 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.190:36871
I20260925 02:47:04.429303 1461 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.190:36871 every 8 connection(s)
I20260925 02:47:04.430255 1395 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
I20260925 02:47:04.430704 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 1395
I20260925 02:47:04.430850 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790304178325258-13942-0/minicluster-data/master-0/wal/instance
I20260925 02:47:04.431209 13942 external_mini_cluster.cc:1428] Setting key b05c8453337cebc4c5457f3a6231727d
I20260925 02:47:04.433740 1462 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:47:04.439930 1462 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 62fa485bead44f2ebb1a9f7e65c47d1a: Bootstrap starting.
I20260925 02:47:04.440634 1465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:04.432550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60266 (local address 127.13.157.190:36871)
0925 02:47:04.433181 (+ 631us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:04.433198 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:04.433236 (+ 38us) server_negotiation.cc:407] Connection header received
0925 02:47:04.433913 (+ 677us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:04.433932 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:04.434320 (+ 388us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:04.434617 (+ 297us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:47:04.434995 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:04.435978 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:04.437183 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:04.437714 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:04.437843 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:04.438737 (+ 894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:47:04.438742 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:47:04.438755 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:47:04.439220 (+ 465us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:47:04.439279 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:47:04.439302 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:47:04.439394 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:04.439586 (+ 192us) server_negotiation.cc:299] Negotiation successful
0925 02:47:04.439848 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":290,"thread_start_us":135,"threads_started":1}
I20260925 02:47:04.442303 1462 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 62fa485bead44f2ebb1a9f7e65c47d1a: Neither blocks nor log segments found. Creating new log.
I20260925 02:47:04.443204 1462 log.cc:824] T 00000000000000000000000000000000 P 62fa485bead44f2ebb1a9f7e65c47d1a: Log is configured to *not* fsync() on all Append() calls
I20260925 02:47:04.445067 1462 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 62fa485bead44f2ebb1a9f7e65c47d1a: No bootstrap required, opened a new log
I20260925 02:47:04.448767 1462 raft_consensus.cc:348] T 00000000000000000000000000000000 P 62fa485bead44f2ebb1a9f7e65c47d1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62fa485bead44f2ebb1a9f7e65c47d1a" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 36871 } }
I20260925 02:47:04.449080 1462 raft_consensus.cc:374] T 00000000000000000000000000000000 P 62fa485bead44f2ebb1a9f7e65c47d1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:47:04.449147 1462 raft_consensus.cc:729] T 00000000000000000000000000000000 P 62fa485bead44f2ebb1a9f7e65c47d1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62fa485bead44f2ebb1a9f7e65c47d1a, State: Initialized, Role: FOLLOWER
I20260925 02:47:04.449548 1462 consensus_queue.cc:260] T 00000000000000000000000000000000 P 62fa485bead44f2ebb1a9f7e65c47d1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62fa485bead44f2ebb1a9f7e65c47d1a" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 36871 } }
I20260925 02:47:04.449672 1462 raft_consensus.cc:388] T 00000000000000000000000000000000 P 62fa485bead44f2ebb1a9f7e65c47d1a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 02:47:04.449717 1462 raft_consensus.cc:482] T 00000000000000000000000000000000 P 62fa485bead44f2ebb1a9f7e65c47d1a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 02:47:04.449787 1462 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 62fa485bead44f2ebb1a9f7e65c47d1a [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:47:04.450623 1462 raft_consensus.cc:504] T 00000000000000000000000000000000 P 62fa485bead44f2ebb1a9f7e65c47d1a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62fa485bead44f2ebb1a9f7e65c47d1a" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 36871 } }
I20260925 02:47:04.450963 1462 leader_election.cc:302] T 00000000000000000000000000000000 P 62fa485bead44f2ebb1a9f7e65c47d1a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 62fa485bead44f2ebb1a9f7e65c47d1a; no voters:
I20260925 02:47:04.451237 1462 leader_election.cc:288] T 00000000000000000000000000000000 P 62fa485bead44f2ebb1a9f7e65c47d1a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 02:47:04.451350 1467 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 62fa485bead44f2ebb1a9f7e65c47d1a [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:47:04.451586 1467 raft_consensus.cc:686] T 00000000000000000000000000000000 P 62fa485bead44f2ebb1a9f7e65c47d1a [term 1 LEADER]: Becoming Leader. State: Replica: 62fa485bead44f2ebb1a9f7e65c47d1a, State: Running, Role: LEADER
I20260925 02:47:04.451889 1467 consensus_queue.cc:237] T 00000000000000000000000000000000 P 62fa485bead44f2ebb1a9f7e65c47d1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62fa485bead44f2ebb1a9f7e65c47d1a" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 36871 } }
I20260925 02:47:04.452373 1462 sys_catalog.cc:564] T 00000000000000000000000000000000 P 62fa485bead44f2ebb1a9f7e65c47d1a [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:47:04.453449 1468 sys_catalog.cc:455] T 00000000000000000000000000000000 P 62fa485bead44f2ebb1a9f7e65c47d1a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "62fa485bead44f2ebb1a9f7e65c47d1a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62fa485bead44f2ebb1a9f7e65c47d1a" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 36871 } } }
I20260925 02:47:04.453577 1468 sys_catalog.cc:458] T 00000000000000000000000000000000 P 62fa485bead44f2ebb1a9f7e65c47d1a [sys.catalog]: This master's current role is: LEADER
I20260925 02:47:04.453500 1469 sys_catalog.cc:455] T 00000000000000000000000000000000 P 62fa485bead44f2ebb1a9f7e65c47d1a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 62fa485bead44f2ebb1a9f7e65c47d1a. Latest consensus state: current_term: 1 leader_uuid: "62fa485bead44f2ebb1a9f7e65c47d1a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62fa485bead44f2ebb1a9f7e65c47d1a" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 36871 } } }
I20260925 02:47:04.453751 1469 sys_catalog.cc:458] T 00000000000000000000000000000000 P 62fa485bead44f2ebb1a9f7e65c47d1a [sys.catalog]: This master's current role is: LEADER
I20260925 02:47:04.454005 1476 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:47:04.457086 1476 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:47:04.462754 1476 catalog_manager.cc:1347] Generated new cluster ID: ef6fb2677a53403cbeca2ab0b3d72475
I20260925 02:47:04.462811 1476 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 02:47:04.479815 1476 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 02:47:04.480696 1476 catalog_manager.cc:1504] Loading token signing keys...
I20260925 02:47:04.491483 1476 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 62fa485bead44f2ebb1a9f7e65c47d1a: Generated new TSK 0
I20260925 02:47:04.492143 1476 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 02:47:04.500770 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790304178325258-13942-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790304178325258-13942-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790304178325258-13942-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790304178325258-13942-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.157.129:0
--local_ip_for_outbound_sockets=127.13.157.129
--webserver_interface=127.13.157.129
--webserver_port=0
--tserver_master_addrs=127.13.157.190:36871
--builtin_ntp_servers=127.13.157.148:43429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260925 02:47:04.610229 1486 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:47:04.610545 1486 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:47:04.610586 1486 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:47:04.614511 1486 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:47:04.614558 1486 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:47:04.614610 1486 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.157.129
I20260925 02:47:04.619288 1486 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:43429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790304178325258-13942-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790304178325258-13942-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.157.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790304178325258-13942-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.157.129
--webserver_port=0
--tserver_master_addrs=127.13.157.190:36871
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1486
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.157.129
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790304178325258-13942-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:47:04.620253 1486 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:47:04.621559 1486 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:47:04.630725 1492 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:47:04.630698 1491 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:47:04.630698 1494 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:47:04.630983 1486 server_base.cc:1034] running on GCE node
I20260925 02:47:04.631409 1486 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:47:04.631981 1486 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:47:04.633150 1486 hybrid_clock.cc:648] HybridClock initialized: now 1790304424633133 us; error 27 us; skew 500 ppm
I20260925 02:47:04.634847 1486 webserver.cc:460] Webserver started at http://127.13.157.129:37823/ using document root <none> and password file <none>
I20260925 02:47:04.635417 1486 fs_manager.cc:359] Metadata directory not provided
I20260925 02:47:04.635485 1486 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:47:04.635677 1486 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:47:04.637655 1486 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790304178325258-13942-0/minicluster-data/ts-0/data/instance:
uuid: "ce3df7bf1e3d48248b12ec44683bd72b"
format_stamp: "Formatted at 2026-09-25 02:47:04 on dist-test-slave-kznw"
server_key: "0add65c3148b77585bfe0850ffad1bb6"
server_key_iv: "27abd428a82221c7cda5c7d906610907"
server_key_version: "encryptionkey@0"
I20260925 02:47:04.638192 1486 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790304178325258-13942-0/minicluster-data/ts-0/wal/instance:
uuid: "ce3df7bf1e3d48248b12ec44683bd72b"
format_stamp: "Formatted at 2026-09-25 02:47:04 on dist-test-slave-kznw"
server_key: "0add65c3148b77585bfe0850ffad1bb6"
server_key_iv: "27abd428a82221c7cda5c7d906610907"
server_key_version: "encryptionkey@0"
I20260925 02:47:04.641927 1486 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260925 02:47:04.644582 1500 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:47:04.645857 1486 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260925 02:47:04.645978 1486 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790304178325258-13942-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790304178325258-13942-0/minicluster-data/ts-0/wal
uuid: "ce3df7bf1e3d48248b12ec44683bd72b"
format_stamp: "Formatted at 2026-09-25 02:47:04 on dist-test-slave-kznw"
server_key: "0add65c3148b77585bfe0850ffad1bb6"
server_key_iv: "27abd428a82221c7cda5c7d906610907"
server_key_version: "encryptionkey@0"
I20260925 02:47:04.646103 1486 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790304178325258-13942-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790304178325258-13942-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790304178325258-13942-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:47:04.664337 1486 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:47:04.665105 1486 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:47:04.665292 1486 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:47:04.665917 1486 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:47:04.666899 1486 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:47:04.666970 1486 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:47:04.667037 1486 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:47:04.667061 1486 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:47:04.675544 1486 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.129:36797
I20260925 02:47:04.675580 1613 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.129:36797 every 8 connection(s)
I20260925 02:47:04.676455 1486 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790304178325258-13942-0/minicluster-data/ts-0/data/info.pb
I20260925 02:47:04.676743 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 1486
I20260925 02:47:04.676852 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790304178325258-13942-0/minicluster-data/ts-0/wal/instance
I20260925 02:47:04.677211 13942 external_mini_cluster.cc:1428] Setting key 20f74fe93ea15d7271d4227ad587319c
I20260925 02:47:04.679490 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790304178325258-13942-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790304178325258-13942-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790304178325258-13942-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790304178325258-13942-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.157.130:0
--local_ip_for_outbound_sockets=127.13.157.130
--webserver_interface=127.13.157.130
--webserver_port=0
--tserver_master_addrs=127.13.157.190:36871
--builtin_ntp_servers=127.13.157.148:43429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260925 02:47:04.686064 1465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:04.678247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.129:42621 (local address 127.13.157.190:36871)
0925 02:47:04.678386 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:04.678391 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:04.679711 (+ 1320us) server_negotiation.cc:407] Connection header received
0925 02:47:04.681408 (+ 1697us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:04.681412 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:04.681474 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:04.681547 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:47:04.682471 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:04.683040 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:04.684185 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:04.684421 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:04.684508 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:04.685559 (+ 1051us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:47:04.685563 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:47:04.685566 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:47:04.685581 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:47:04.685602 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:47:04.685610 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:47:04.685667 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:04.685805 (+ 138us) server_negotiation.cc:299] Negotiation successful
0925 02:47:04.685898 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260925 02:47:04.686954 1616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:04.678739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:36871 (local address 127.13.157.129:42621)
0925 02:47:04.679559 (+ 820us) negotiation.cc:107] Waiting for socket to connect
0925 02:47:04.679618 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0925 02:47:04.681200 (+ 1582us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:47:04.681736 (+ 536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:47:04.681743 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:47:04.681834 (+ 91us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:47:04.682332 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:47:04.682346 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:47:04.683164 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:47:04.683168 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:47:04.684069 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:47:04.684079 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:47:04.684574 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:47:04.684579 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:47:04.684741 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:04.685304 (+ 563us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 02:47:04.685306 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:47:04.685348 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 02:47:04.685413 (+ 65us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:47:04.685677 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:47:04.685686 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:47:04.685693 (+ 7us) client_negotiation.cc:769] Sending connection context
0925 02:47:04.685800 (+ 107us) client_negotiation.cc:240] Negotiation successful
0925 02:47:04.686093 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":159,"threads_started":1}
I20260925 02:47:04.688333 1614 heartbeater.cc:347] Connected to a master server at 127.13.157.190:36871
I20260925 02:47:04.688621 1614 heartbeater.cc:464] Registering TS with master...
I20260925 02:47:04.689191 1614 heartbeater.cc:511] Master 127.13.157.190:36871 requested a full tablet report, sending...
I20260925 02:47:04.690881 1426 ts_manager.cc:194] Registered new tserver with Master: ce3df7bf1e3d48248b12ec44683bd72b (127.13.157.129:36797)
I20260925 02:47:04.692493 1426 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.157.129:42621
W20260925 02:47:04.790145 1617 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:47:04.790380 1617 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:47:04.790414 1617 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:47:04.794275 1617 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:47:04.794325 1617 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:47:04.794375 1617 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.157.130
I20260925 02:47:04.798934 1617 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:43429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790304178325258-13942-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790304178325258-13942-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.157.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790304178325258-13942-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.157.130
--webserver_port=0
--tserver_master_addrs=127.13.157.190:36871
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1617
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.157.130
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790304178325258-13942-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:47:04.799911 1617 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:47:04.801263 1617 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:47:04.810600 1623 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:47:04.810735 1617 server_base.cc:1034] running on GCE node
W20260925 02:47:04.810606 1625 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:47:04.810618 1622 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:47:04.811228 1617 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:47:04.811847 1617 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:47:04.813015 1617 hybrid_clock.cc:648] HybridClock initialized: now 1790304424812996 us; error 33 us; skew 500 ppm
I20260925 02:47:04.814806 1617 webserver.cc:460] Webserver started at http://127.13.157.130:42201/ using document root <none> and password file <none>
I20260925 02:47:04.815398 1617 fs_manager.cc:359] Metadata directory not provided
I20260925 02:47:04.815467 1617 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:47:04.815676 1617 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:47:04.817672 1617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790304178325258-13942-0/minicluster-data/ts-1/data/instance:
uuid: "b051b8d15cc34d3091f2b4069b86ba68"
format_stamp: "Formatted at 2026-09-25 02:47:04 on dist-test-slave-kznw"
server_key: "f6b29e74854a848e48cbde2d21b399aa"
server_key_iv: "57ce4bd7776b91dd5af6352388347a5e"
server_key_version: "encryptionkey@0"
I20260925 02:47:04.818218 1617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790304178325258-13942-0/minicluster-data/ts-1/wal/instance:
uuid: "b051b8d15cc34d3091f2b4069b86ba68"
format_stamp: "Formatted at 2026-09-25 02:47:04 on dist-test-slave-kznw"
server_key: "f6b29e74854a848e48cbde2d21b399aa"
server_key_iv: "57ce4bd7776b91dd5af6352388347a5e"
server_key_version: "encryptionkey@0"
I20260925 02:47:04.821763 1617 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260925 02:47:04.824126 1631 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:47:04.825243 1617 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260925 02:47:04.825345 1617 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790304178325258-13942-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790304178325258-13942-0/minicluster-data/ts-1/wal
uuid: "b051b8d15cc34d3091f2b4069b86ba68"
format_stamp: "Formatted at 2026-09-25 02:47:04 on dist-test-slave-kznw"
server_key: "f6b29e74854a848e48cbde2d21b399aa"
server_key_iv: "57ce4bd7776b91dd5af6352388347a5e"
server_key_version: "encryptionkey@0"
I20260925 02:47:04.825434 1617 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790304178325258-13942-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790304178325258-13942-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790304178325258-13942-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:47:04.834463 1617 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:47:04.835098 1617 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:47:04.835253 1617 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:47:04.835808 1617 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:47:04.836689 1617 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:47:04.836736 1617 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:47:04.836787 1617 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:47:04.836818 1617 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:47:04.845487 1617 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.130:34025
I20260925 02:47:04.845517 1744 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.130:34025 every 8 connection(s)
I20260925 02:47:04.846448 1617 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790304178325258-13942-0/minicluster-data/ts-1/data/info.pb
I20260925 02:47:04.855726 1465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:04.848559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.130:50915 (local address 127.13.157.190:36871)
0925 02:47:04.848699 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:04.848703 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:04.849795 (+ 1092us) server_negotiation.cc:407] Connection header received
0925 02:47:04.851193 (+ 1398us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:04.851196 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:04.851248 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:04.851320 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:47:04.852178 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:04.852787 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:04.853942 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:04.854161 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:04.854238 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:04.855260 (+ 1022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:47:04.855263 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:47:04.855265 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:47:04.855278 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:47:04.855297 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:47:04.855305 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:47:04.855371 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:04.855506 (+ 135us) server_negotiation.cc:299] Negotiation successful
0925 02:47:04.855557 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260925 02:47:04.855913 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 1617
I20260925 02:47:04.856045 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790304178325258-13942-0/minicluster-data/ts-1/wal/instance
I20260925 02:47:04.856326 13942 external_mini_cluster.cc:1428] Setting key dc98b45eaf60aea462e1f4070b99b380
I20260925 02:47:04.856621 1747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:04.848953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:36871 (local address 127.13.157.130:50915)
0925 02:47:04.849635 (+ 682us) negotiation.cc:107] Waiting for socket to connect
0925 02:47:04.849676 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0925 02:47:04.850990 (+ 1314us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:47:04.851455 (+ 465us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:47:04.851462 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:47:04.851553 (+ 91us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:47:04.852040 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:47:04.852073 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:47:04.852908 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:47:04.852912 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:47:04.853830 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:47:04.853842 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:47:04.854281 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:47:04.854284 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:47:04.854418 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:04.854999 (+ 581us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 02:47:04.855001 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:47:04.855044 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 02:47:04.855106 (+ 62us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:47:04.855378 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:47:04.855384 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:47:04.855388 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 02:47:04.855480 (+ 92us) client_negotiation.cc:240] Negotiation successful
0925 02:47:04.855707 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":116,"threads_started":1}
I20260925 02:47:04.857676 1745 heartbeater.cc:347] Connected to a master server at 127.13.157.190:36871
I20260925 02:47:04.857893 1745 heartbeater.cc:464] Registering TS with master...
I20260925 02:47:04.858230 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790304178325258-13942-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790304178325258-13942-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790304178325258-13942-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790304178325258-13942-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.157.131:0
--local_ip_for_outbound_sockets=127.13.157.131
--webserver_interface=127.13.157.131
--webserver_port=0
--tserver_master_addrs=127.13.157.190:36871
--builtin_ntp_servers=127.13.157.148:43429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260925 02:47:04.858382 1745 heartbeater.cc:511] Master 127.13.157.190:36871 requested a full tablet report, sending...
I20260925 02:47:04.859445 1426 ts_manager.cc:194] Registered new tserver with Master: b051b8d15cc34d3091f2b4069b86ba68 (127.13.157.130:34025)
I20260925 02:47:04.860034 1426 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.157.130:50915
W20260925 02:47:04.968439 1748 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:47:04.968678 1748 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:47:04.968714 1748 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:47:04.972572 1748 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:47:04.972617 1748 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:47:04.972666 1748 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.157.131
I20260925 02:47:04.977270 1748 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:43429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790304178325258-13942-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790304178325258-13942-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.157.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790304178325258-13942-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.157.131
--webserver_port=0
--tserver_master_addrs=127.13.157.190:36871
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1748
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.157.131
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790304178325258-13942-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:47:04.978227 1748 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:47:04.979592 1748 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:47:04.988641 1754 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:47:04.988710 1748 server_base.cc:1034] running on GCE node
W20260925 02:47:04.988631 1753 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:47:04.988651 1756 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:47:04.989279 1748 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:47:04.989866 1748 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:47:04.990978 1748 hybrid_clock.cc:648] HybridClock initialized: now 1790304424990953 us; error 38 us; skew 500 ppm
I20260925 02:47:04.992633 1748 webserver.cc:460] Webserver started at http://127.13.157.131:41379/ using document root <none> and password file <none>
I20260925 02:47:04.993170 1748 fs_manager.cc:359] Metadata directory not provided
I20260925 02:47:04.993234 1748 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:47:04.993415 1748 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:47:04.995368 1748 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790304178325258-13942-0/minicluster-data/ts-2/data/instance:
uuid: "039de77fe17441f583166d443559d043"
format_stamp: "Formatted at 2026-09-25 02:47:04 on dist-test-slave-kznw"
server_key: "4b98f33f3168448a11b8ea40f48ff217"
server_key_iv: "dc8d8f5c7775fa86b88fd55ad5b26b02"
server_key_version: "encryptionkey@0"
I20260925 02:47:04.995890 1748 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790304178325258-13942-0/minicluster-data/ts-2/wal/instance:
uuid: "039de77fe17441f583166d443559d043"
format_stamp: "Formatted at 2026-09-25 02:47:04 on dist-test-slave-kznw"
server_key: "4b98f33f3168448a11b8ea40f48ff217"
server_key_iv: "dc8d8f5c7775fa86b88fd55ad5b26b02"
server_key_version: "encryptionkey@0"
I20260925 02:47:04.999333 1748 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260925 02:47:05.001735 1762 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:47:05.003048 1748 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260925 02:47:05.003145 1748 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790304178325258-13942-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790304178325258-13942-0/minicluster-data/ts-2/wal
uuid: "039de77fe17441f583166d443559d043"
format_stamp: "Formatted at 2026-09-25 02:47:04 on dist-test-slave-kznw"
server_key: "4b98f33f3168448a11b8ea40f48ff217"
server_key_iv: "dc8d8f5c7775fa86b88fd55ad5b26b02"
server_key_version: "encryptionkey@0"
I20260925 02:47:05.003244 1748 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790304178325258-13942-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790304178325258-13942-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790304178325258-13942-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:47:05.024950 1748 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:47:05.025707 1748 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:47:05.025877 1748 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:47:05.026553 1748 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:47:05.027527 1748 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:47:05.027581 1748 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:47:05.027626 1748 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:47:05.027654 1748 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:47:05.036000 1748 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.131:46157
I20260925 02:47:05.036032 1875 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.131:46157 every 8 connection(s)
I20260925 02:47:05.036864 1748 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790304178325258-13942-0/minicluster-data/ts-2/data/info.pb
I20260925 02:47:05.045049 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 1748
I20260925 02:47:05.045154 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790304178325258-13942-0/minicluster-data/ts-2/wal/instance
I20260925 02:47:05.045457 13942 external_mini_cluster.cc:1428] Setting key 61b2d9151b426ea03b92c06adea5d83d
I20260925 02:47:05.045892 1465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:05.038746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.131:45241 (local address 127.13.157.190:36871)
0925 02:47:05.038865 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:05.038868 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:05.039932 (+ 1064us) server_negotiation.cc:407] Connection header received
0925 02:47:05.041304 (+ 1372us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:05.041307 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:05.041367 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:05.041440 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:47:05.042347 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:05.042868 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:05.044047 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:05.044262 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:05.044351 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:05.045387 (+ 1036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:47:05.045391 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:47:05.045394 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:47:05.045409 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:47:05.045433 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:47:05.045444 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:47:05.045532 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:05.045704 (+ 172us) server_negotiation.cc:299] Negotiation successful
0925 02:47:05.045770 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260925 02:47:05.046859 1878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:05.039172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:36871 (local address 127.13.157.131:45241)
0925 02:47:05.039788 (+ 616us) negotiation.cc:107] Waiting for socket to connect
0925 02:47:05.039828 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0925 02:47:05.041099 (+ 1271us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:47:05.041633 (+ 534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:47:05.041642 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:47:05.041735 (+ 93us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:47:05.042211 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:47:05.042224 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:47:05.043022 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:47:05.043025 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:47:05.043931 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:47:05.043940 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:47:05.044397 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:47:05.044401 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:47:05.044560 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:05.045107 (+ 547us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 02:47:05.045109 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:47:05.045149 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 02:47:05.045229 (+ 80us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:47:05.045564 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:47:05.045572 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:47:05.045578 (+ 6us) client_negotiation.cc:769] Sending connection context
0925 02:47:05.045681 (+ 103us) client_negotiation.cc:240] Negotiation successful
0925 02:47:05.045949 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":105,"threads_started":1}
I20260925 02:47:05.047917 1876 heartbeater.cc:347] Connected to a master server at 127.13.157.190:36871
I20260925 02:47:05.048132 1876 heartbeater.cc:464] Registering TS with master...
I20260925 02:47:05.048609 1876 heartbeater.cc:511] Master 127.13.157.190:36871 requested a full tablet report, sending...
I20260925 02:47:05.049547 1425 ts_manager.cc:194] Registered new tserver with Master: 039de77fe17441f583166d443559d043 (127.13.157.131:46157)
I20260925 02:47:05.050105 1425 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.157.131:45241
I20260925 02:47:05.059439 13942 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 02:47:05.064563 1465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:05.061312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60274 (local address 127.13.157.190:36871)
0925 02:47:05.061411 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:05.061415 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:05.061634 (+ 219us) server_negotiation.cc:407] Connection header received
0925 02:47:05.061725 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:05.061727 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:05.061765 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:05.061823 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:47:05.062027 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:05.062532 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:05.063321 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:05.063541 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:05.063615 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:05.064259 (+ 644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:47:05.064261 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:47:05.064263 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:47:05.064272 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:47:05.064285 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:47:05.064291 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:47:05.064337 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:05.064393 (+ 56us) server_negotiation.cc:299] Negotiation successful
0925 02:47:05.064435 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260925 02:47:05.067261 1425 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60274:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 02:47:05.070302 1425 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 02:47:05.084046 1888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:05.078446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.129:36797 (local address 127.0.0.1:52244)
0925 02:47:05.078965 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0925 02:47:05.079013 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0925 02:47:05.079152 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:47:05.080131 (+ 979us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:47:05.080136 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:47:05.080158 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:47:05.080281 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:47:05.080288 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:47:05.081519 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:47:05.081522 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:47:05.082644 (+ 1122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:47:05.082654 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:47:05.083518 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:47:05.083522 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:47:05.083629 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:05.083669 (+ 40us) client_negotiation.cc:769] Sending connection context
0925 02:47:05.083771 (+ 102us) client_negotiation.cc:240] Negotiation successful
0925 02:47:05.083888 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":86,"threads_started":1}
I20260925 02:47:05.084102 1889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:05.078601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52244 (local address 127.13.157.129:36797)
0925 02:47:05.079070 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:05.079083 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:05.079109 (+ 26us) server_negotiation.cc:407] Connection header received
0925 02:47:05.079727 (+ 618us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:05.079733 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:05.079981 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:05.080129 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:47:05.080647 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:05.081413 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:05.082760 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:05.083391 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:05.083512 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:05.083687 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:05.083823 (+ 136us) server_negotiation.cc:299] Negotiation successful
0925 02:47:05.083932 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":376,"thread_start_us":108,"threads_started":1}
I20260925 02:47:05.084605 1887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:05.078597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.131:46157 (local address 127.0.0.1:42822)
0925 02:47:05.079373 (+ 776us) negotiation.cc:107] Waiting for socket to connect
0925 02:47:05.079386 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 02:47:05.079456 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:47:05.080388 (+ 932us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:47:05.080391 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:47:05.080417 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:47:05.080532 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:47:05.080538 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:47:05.081953 (+ 1415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:47:05.081957 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:47:05.083176 (+ 1219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:47:05.083187 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:47:05.084209 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:47:05.084213 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:47:05.084318 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:05.084342 (+ 24us) client_negotiation.cc:769] Sending connection context
0925 02:47:05.084385 (+ 43us) client_negotiation.cc:240] Negotiation successful
0925 02:47:05.084458 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":717,"mutex_wait_us":19,"spinlock_wait_cycles":1280,"thread_start_us":100,"threads_started":1}
I20260925 02:47:05.084875 1890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:05.079239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.130:34025 (local address 127.0.0.1:39362)
0925 02:47:05.079627 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0925 02:47:05.079639 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 02:47:05.079713 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:47:05.080396 (+ 683us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:47:05.080399 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:47:05.080430 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:47:05.080523 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:47:05.080530 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:47:05.082110 (+ 1580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:47:05.082113 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:47:05.083214 (+ 1101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:47:05.083222 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:47:05.084187 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:47:05.084191 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:47:05.084289 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:05.084314 (+ 25us) client_negotiation.cc:769] Sending connection context
0925 02:47:05.084362 (+ 48us) client_negotiation.cc:240] Negotiation successful
0925 02:47:05.084428 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":333,"mutex_wait_us":1,"spinlock_wait_cycles":2176,"thread_start_us":92,"threads_started":1}
I20260925 02:47:05.085018 1892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:05.079688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39362 (local address 127.13.157.130:34025)
0925 02:47:05.079980 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:05.080001 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:05.080019 (+ 18us) server_negotiation.cc:407] Connection header received
0925 02:47:05.080079 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:05.080083 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:05.080250 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:05.080399 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:47:05.080625 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:05.081843 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:05.083330 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:05.084056 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:05.084553 (+ 497us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:05.084717 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:05.084768 (+ 51us) server_negotiation.cc:299] Negotiation successful
0925 02:47:05.084853 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":202,"thread_start_us":99,"threads_started":1}
I20260925 02:47:05.085354 1891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:05.079381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42822 (local address 127.13.157.131:46157)
0925 02:47:05.079957 (+ 576us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:05.079971 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:05.079993 (+ 22us) server_negotiation.cc:407] Connection header received
0925 02:47:05.080070 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:05.080075 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:05.080250 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:05.080399 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:47:05.080625 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:05.081819 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:05.083301 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:05.084067 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:05.084896 (+ 829us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:05.085063 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:05.085116 (+ 53us) server_negotiation.cc:299] Negotiation successful
0925 02:47:05.085204 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":477,"thread_start_us":122,"threads_started":1}
I20260925 02:47:05.087592 1810 tablet_service.cc:1467] Processing CreateTablet for tablet 05cc59745b4d4a1094a8266a389f85ee (DEFAULT_TABLE table=test-table [id=c2601cb498d64d4eb557b2531c92c085]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:47:05.087610 1548 tablet_service.cc:1467] Processing CreateTablet for tablet 05cc59745b4d4a1094a8266a389f85ee (DEFAULT_TABLE table=test-table [id=c2601cb498d64d4eb557b2531c92c085]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:47:05.087610 1679 tablet_service.cc:1467] Processing CreateTablet for tablet 05cc59745b4d4a1094a8266a389f85ee (DEFAULT_TABLE table=test-table [id=c2601cb498d64d4eb557b2531c92c085]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:47:05.088559 1810 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05cc59745b4d4a1094a8266a389f85ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:47:05.088603 1679 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05cc59745b4d4a1094a8266a389f85ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:47:05.088603 1548 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05cc59745b4d4a1094a8266a389f85ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:47:05.094712 1893 tablet_bootstrap.cc:492] T 05cc59745b4d4a1094a8266a389f85ee P 039de77fe17441f583166d443559d043: Bootstrap starting.
I20260925 02:47:05.096257 1894 tablet_bootstrap.cc:492] T 05cc59745b4d4a1094a8266a389f85ee P ce3df7bf1e3d48248b12ec44683bd72b: Bootstrap starting.
I20260925 02:47:05.096735 1895 tablet_bootstrap.cc:492] T 05cc59745b4d4a1094a8266a389f85ee P b051b8d15cc34d3091f2b4069b86ba68: Bootstrap starting.
I20260925 02:47:05.097128 1893 tablet_bootstrap.cc:654] T 05cc59745b4d4a1094a8266a389f85ee P 039de77fe17441f583166d443559d043: Neither blocks nor log segments found. Creating new log.
I20260925 02:47:05.097980 1893 log.cc:824] T 05cc59745b4d4a1094a8266a389f85ee P 039de77fe17441f583166d443559d043: Log is configured to *not* fsync() on all Append() calls
I20260925 02:47:05.098484 1895 tablet_bootstrap.cc:654] T 05cc59745b4d4a1094a8266a389f85ee P b051b8d15cc34d3091f2b4069b86ba68: Neither blocks nor log segments found. Creating new log.
I20260925 02:47:05.098516 1894 tablet_bootstrap.cc:654] T 05cc59745b4d4a1094a8266a389f85ee P ce3df7bf1e3d48248b12ec44683bd72b: Neither blocks nor log segments found. Creating new log.
I20260925 02:47:05.099258 1895 log.cc:824] T 05cc59745b4d4a1094a8266a389f85ee P b051b8d15cc34d3091f2b4069b86ba68: Log is configured to *not* fsync() on all Append() calls
I20260925 02:47:05.099323 1894 log.cc:824] T 05cc59745b4d4a1094a8266a389f85ee P ce3df7bf1e3d48248b12ec44683bd72b: Log is configured to *not* fsync() on all Append() calls
I20260925 02:47:05.100315 1893 tablet_bootstrap.cc:492] T 05cc59745b4d4a1094a8266a389f85ee P 039de77fe17441f583166d443559d043: No bootstrap required, opened a new log
I20260925 02:47:05.100500 1893 ts_tablet_manager.cc:1397] T 05cc59745b4d4a1094a8266a389f85ee P 039de77fe17441f583166d443559d043: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260925 02:47:05.101250 1894 tablet_bootstrap.cc:492] T 05cc59745b4d4a1094a8266a389f85ee P ce3df7bf1e3d48248b12ec44683bd72b: No bootstrap required, opened a new log
I20260925 02:47:05.101250 1895 tablet_bootstrap.cc:492] T 05cc59745b4d4a1094a8266a389f85ee P b051b8d15cc34d3091f2b4069b86ba68: No bootstrap required, opened a new log
I20260925 02:47:05.101438 1895 ts_tablet_manager.cc:1397] T 05cc59745b4d4a1094a8266a389f85ee P b051b8d15cc34d3091f2b4069b86ba68: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260925 02:47:05.101432 1894 ts_tablet_manager.cc:1397] T 05cc59745b4d4a1094a8266a389f85ee P ce3df7bf1e3d48248b12ec44683bd72b: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260925 02:47:05.103783 1893 raft_consensus.cc:348] T 05cc59745b4d4a1094a8266a389f85ee P 039de77fe17441f583166d443559d043 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce3df7bf1e3d48248b12ec44683bd72b" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 36797 } } peers { permanent_uuid: "039de77fe17441f583166d443559d043" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 46157 } } peers { permanent_uuid: "b051b8d15cc34d3091f2b4069b86ba68" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 34025 } }
I20260925 02:47:05.104013 1893 raft_consensus.cc:374] T 05cc59745b4d4a1094a8266a389f85ee P 039de77fe17441f583166d443559d043 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:47:05.104065 1893 raft_consensus.cc:729] T 05cc59745b4d4a1094a8266a389f85ee P 039de77fe17441f583166d443559d043 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 039de77fe17441f583166d443559d043, State: Initialized, Role: FOLLOWER
I20260925 02:47:05.104470 1893 consensus_queue.cc:260] T 05cc59745b4d4a1094a8266a389f85ee P 039de77fe17441f583166d443559d043 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce3df7bf1e3d48248b12ec44683bd72b" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 36797 } } peers { permanent_uuid: "039de77fe17441f583166d443559d043" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 46157 } } peers { permanent_uuid: "b051b8d15cc34d3091f2b4069b86ba68" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 34025 } }
I20260925 02:47:05.105079 1876 heartbeater.cc:503] Master 127.13.157.190:36871 was elected leader, sending a full tablet report...
I20260925 02:47:05.105463 1893 ts_tablet_manager.cc:1428] T 05cc59745b4d4a1094a8266a389f85ee P 039de77fe17441f583166d443559d043: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260925 02:47:05.105564 1894 raft_consensus.cc:348] T 05cc59745b4d4a1094a8266a389f85ee P ce3df7bf1e3d48248b12ec44683bd72b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce3df7bf1e3d48248b12ec44683bd72b" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 36797 } } peers { permanent_uuid: "039de77fe17441f583166d443559d043" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 46157 } } peers { permanent_uuid: "b051b8d15cc34d3091f2b4069b86ba68" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 34025 } }
I20260925 02:47:05.105564 1895 raft_consensus.cc:348] T 05cc59745b4d4a1094a8266a389f85ee P b051b8d15cc34d3091f2b4069b86ba68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce3df7bf1e3d48248b12ec44683bd72b" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 36797 } } peers { permanent_uuid: "039de77fe17441f583166d443559d043" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 46157 } } peers { permanent_uuid: "b051b8d15cc34d3091f2b4069b86ba68" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 34025 } }
I20260925 02:47:05.105808 1894 raft_consensus.cc:374] T 05cc59745b4d4a1094a8266a389f85ee P ce3df7bf1e3d48248b12ec44683bd72b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:47:05.105808 1895 raft_consensus.cc:374] T 05cc59745b4d4a1094a8266a389f85ee P b051b8d15cc34d3091f2b4069b86ba68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:47:05.105865 1895 raft_consensus.cc:729] T 05cc59745b4d4a1094a8266a389f85ee P b051b8d15cc34d3091f2b4069b86ba68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b051b8d15cc34d3091f2b4069b86ba68, State: Initialized, Role: FOLLOWER
I20260925 02:47:05.105865 1894 raft_consensus.cc:729] T 05cc59745b4d4a1094a8266a389f85ee P ce3df7bf1e3d48248b12ec44683bd72b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce3df7bf1e3d48248b12ec44683bd72b, State: Initialized, Role: FOLLOWER
I20260925 02:47:05.106321 1895 consensus_queue.cc:260] T 05cc59745b4d4a1094a8266a389f85ee P b051b8d15cc34d3091f2b4069b86ba68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce3df7bf1e3d48248b12ec44683bd72b" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 36797 } } peers { permanent_uuid: "039de77fe17441f583166d443559d043" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 46157 } } peers { permanent_uuid: "b051b8d15cc34d3091f2b4069b86ba68" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 34025 } }
I20260925 02:47:05.106321 1894 consensus_queue.cc:260] T 05cc59745b4d4a1094a8266a389f85ee P ce3df7bf1e3d48248b12ec44683bd72b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce3df7bf1e3d48248b12ec44683bd72b" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 36797 } } peers { permanent_uuid: "039de77fe17441f583166d443559d043" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 46157 } } peers { permanent_uuid: "b051b8d15cc34d3091f2b4069b86ba68" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 34025 } }
I20260925 02:47:05.107316 1895 ts_tablet_manager.cc:1428] T 05cc59745b4d4a1094a8266a389f85ee P b051b8d15cc34d3091f2b4069b86ba68: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260925 02:47:05.107499 1894 ts_tablet_manager.cc:1428] T 05cc59745b4d4a1094a8266a389f85ee P ce3df7bf1e3d48248b12ec44683bd72b: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260925 02:47:05.107681 1745 heartbeater.cc:503] Master 127.13.157.190:36871 was elected leader, sending a full tablet report...
I20260925 02:47:05.107781 1614 heartbeater.cc:503] Master 127.13.157.190:36871 was elected leader, sending a full tablet report...
I20260925 02:47:05.142645 1900 raft_consensus.cc:482] T 05cc59745b4d4a1094a8266a389f85ee P b051b8d15cc34d3091f2b4069b86ba68 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 02:47:05.142836 1900 raft_consensus.cc:504] T 05cc59745b4d4a1094a8266a389f85ee P b051b8d15cc34d3091f2b4069b86ba68 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce3df7bf1e3d48248b12ec44683bd72b" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 36797 } } peers { permanent_uuid: "039de77fe17441f583166d443559d043" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 46157 } } peers { permanent_uuid: "b051b8d15cc34d3091f2b4069b86ba68" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 34025 } }
I20260925 02:47:05.144280 1900 leader_election.cc:288] T 05cc59745b4d4a1094a8266a389f85ee P b051b8d15cc34d3091f2b4069b86ba68 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ce3df7bf1e3d48248b12ec44683bd72b (127.13.157.129:36797), 039de77fe17441f583166d443559d043 (127.13.157.131:46157)
I20260925 02:47:05.148290 1889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:05.144315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.130:51933 (local address 127.13.157.129:36797)
0925 02:47:05.144443 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:05.144448 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:05.144523 (+ 75us) server_negotiation.cc:407] Connection header received
0925 02:47:05.144644 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:05.144647 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:05.144706 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:05.144790 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:47:05.145109 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:05.145807 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:05.147229 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:05.147799 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:05.147910 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:05.147961 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:05.148114 (+ 153us) server_negotiation.cc:299] Negotiation successful
0925 02:47:05.148184 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260925 02:47:05.148316 1747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:05.144234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.129:36797 (local address 127.13.157.130:51933)
0925 02:47:05.144425 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0925 02:47:05.144441 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 02:47:05.144548 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:47:05.144844 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:47:05.144847 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:47:05.144867 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:47:05.145004 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:47:05.145013 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:47:05.145920 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:47:05.145922 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:47:05.147117 (+ 1195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:47:05.147128 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:47:05.147917 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:47:05.147920 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:47:05.147994 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:05.148014 (+ 20us) client_negotiation.cc:769] Sending connection context
0925 02:47:05.148089 (+ 75us) client_negotiation.cc:240] Negotiation successful
0925 02:47:05.148178 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"mutex_wait_us":2}
I20260925 02:47:05.148471 1902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:05.144251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.131:46157 (local address 127.13.157.130:42209)
0925 02:47:05.144934 (+ 683us) negotiation.cc:107] Waiting for socket to connect
0925 02:47:05.144947 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 02:47:05.145048 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:47:05.145303 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:47:05.145305 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:47:05.145324 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:47:05.145423 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:47:05.145427 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:47:05.146233 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:47:05.146236 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:47:05.147350 (+ 1114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:47:05.147357 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:47:05.148096 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:47:05.148099 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:47:05.148211 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:05.148231 (+ 20us) client_negotiation.cc:769] Sending connection context
0925 02:47:05.148296 (+ 65us) client_negotiation.cc:240] Negotiation successful
0925 02:47:05.148352 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":619,"mutex_wait_us":281,"thread_start_us":137,"threads_started":1}
I20260925 02:47:05.148502 1891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:05.144352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.157.130:42209 (local address 127.13.157.131:46157)
0925 02:47:05.144471 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:05.144476 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:05.145031 (+ 555us) server_negotiation.cc:407] Connection header received
0925 02:47:05.145137 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:05.145139 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:05.145189 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:05.145262 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:47:05.145518 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:05.146132 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:05.147444 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:05.147982 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:05.148091 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:05.148134 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:05.148318 (+ 184us) server_negotiation.cc:299] Negotiation successful
0925 02:47:05.148391 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260925 02:47:05.148981 1568 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05cc59745b4d4a1094a8266a389f85ee" candidate_uuid: "b051b8d15cc34d3091f2b4069b86ba68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce3df7bf1e3d48248b12ec44683bd72b" is_pre_election: true
I20260925 02:47:05.149215 1830 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05cc59745b4d4a1094a8266a389f85ee" candidate_uuid: "b051b8d15cc34d3091f2b4069b86ba68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039de77fe17441f583166d443559d043" is_pre_election: true
I20260925 02:47:05.149350 1568 raft_consensus.cc:2454] T 05cc59745b4d4a1094a8266a389f85ee P ce3df7bf1e3d48248b12ec44683bd72b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b051b8d15cc34d3091f2b4069b86ba68 in term 0.
I20260925 02:47:05.149487 1830 raft_consensus.cc:2454] T 05cc59745b4d4a1094a8266a389f85ee P 039de77fe17441f583166d443559d043 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b051b8d15cc34d3091f2b4069b86ba68 in term 0.
I20260925 02:47:05.149839 1635 leader_election.cc:302] T 05cc59745b4d4a1094a8266a389f85ee P b051b8d15cc34d3091f2b4069b86ba68 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b051b8d15cc34d3091f2b4069b86ba68, ce3df7bf1e3d48248b12ec44683bd72b; no voters:
I20260925 02:47:05.150108 1900 raft_consensus.cc:2781] T 05cc59745b4d4a1094a8266a389f85ee P b051b8d15cc34d3091f2b4069b86ba68 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 02:47:05.150185 1900 raft_consensus.cc:482] T 05cc59745b4d4a1094a8266a389f85ee P b051b8d15cc34d3091f2b4069b86ba68 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 02:47:05.150228 1900 raft_consensus.cc:3037] T 05cc59745b4d4a1094a8266a389f85ee P b051b8d15cc34d3091f2b4069b86ba68 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:47:05.151415 1900 raft_consensus.cc:504] T 05cc59745b4d4a1094a8266a389f85ee P b051b8d15cc34d3091f2b4069b86ba68 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce3df7bf1e3d48248b12ec44683bd72b" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 36797 } } peers { permanent_uuid: "039de77fe17441f583166d443559d043" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 46157 } } peers { permanent_uuid: "b051b8d15cc34d3091f2b4069b86ba68" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 34025 } }
I20260925 02:47:05.151741 1900 leader_election.cc:288] T 05cc59745b4d4a1094a8266a389f85ee P b051b8d15cc34d3091f2b4069b86ba68 [CANDIDATE]: Term 1 election: Requested vote from peers ce3df7bf1e3d48248b12ec44683bd72b (127.13.157.129:36797), 039de77fe17441f583166d443559d043 (127.13.157.131:46157)
I20260925 02:47:05.152084 1830 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05cc59745b4d4a1094a8266a389f85ee" candidate_uuid: "b051b8d15cc34d3091f2b4069b86ba68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039de77fe17441f583166d443559d043"
I20260925 02:47:05.152191 1830 raft_consensus.cc:3037] T 05cc59745b4d4a1094a8266a389f85ee P 039de77fe17441f583166d443559d043 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:47:05.153196 1830 raft_consensus.cc:2454] T 05cc59745b4d4a1094a8266a389f85ee P 039de77fe17441f583166d443559d043 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b051b8d15cc34d3091f2b4069b86ba68 in term 1.
I20260925 02:47:05.153461 1568 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05cc59745b4d4a1094a8266a389f85ee" candidate_uuid: "b051b8d15cc34d3091f2b4069b86ba68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce3df7bf1e3d48248b12ec44683bd72b"
I20260925 02:47:05.153541 1633 leader_election.cc:302] T 05cc59745b4d4a1094a8266a389f85ee P b051b8d15cc34d3091f2b4069b86ba68 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 039de77fe17441f583166d443559d043, b051b8d15cc34d3091f2b4069b86ba68; no voters:
I20260925 02:47:05.153606 1568 raft_consensus.cc:3037] T 05cc59745b4d4a1094a8266a389f85ee P ce3df7bf1e3d48248b12ec44683bd72b [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:47:05.153697 1900 raft_consensus.cc:2781] T 05cc59745b4d4a1094a8266a389f85ee P b051b8d15cc34d3091f2b4069b86ba68 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:47:05.153913 1900 raft_consensus.cc:686] T 05cc59745b4d4a1094a8266a389f85ee P b051b8d15cc34d3091f2b4069b86ba68 [term 1 LEADER]: Becoming Leader. State: Replica: b051b8d15cc34d3091f2b4069b86ba68, State: Running, Role: LEADER
I20260925 02:47:05.154284 1900 consensus_queue.cc:237] T 05cc59745b4d4a1094a8266a389f85ee P b051b8d15cc34d3091f2b4069b86ba68 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce3df7bf1e3d48248b12ec44683bd72b" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 36797 } } peers { permanent_uuid: "039de77fe17441f583166d443559d043" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 46157 } } peers { permanent_uuid: "b051b8d15cc34d3091f2b4069b86ba68" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 34025 } }
I20260925 02:47:05.154860 1568 raft_consensus.cc:2454] T 05cc59745b4d4a1094a8266a389f85ee P ce3df7bf1e3d48248b12ec44683bd72b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b051b8d15cc34d3091f2b4069b86ba68 in term 1.
I20260925 02:47:05.157212 1424 catalog_manager.cc:5495] T 05cc59745b4d4a1094a8266a389f85ee P b051b8d15cc34d3091f2b4069b86ba68 reported cstate change: term changed from 0 to 1, leader changed from <none> to b051b8d15cc34d3091f2b4069b86ba68 (127.13.157.130). New cstate: current_term: 1 leader_uuid: "b051b8d15cc34d3091f2b4069b86ba68" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce3df7bf1e3d48248b12ec44683bd72b" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 36797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "039de77fe17441f583166d443559d043" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 46157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b051b8d15cc34d3091f2b4069b86ba68" member_type: VOTER last_known_addr { host: "127.13.157.130" port: 34025 } health_report { overall_health: HEALTHY } } }
W20260925 02:47:05.177657 1615 tablet.cc:2369] T 05cc59745b4d4a1094a8266a389f85ee P ce3df7bf1e3d48248b12ec44683bd72b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 02:47:05.181334 1892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:05.177888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39370 (local address 127.13.157.130:34025)
0925 02:47:05.178038 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:05.178042 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:05.178057 (+ 15us) server_negotiation.cc:407] Connection header received
0925 02:47:05.178139 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:05.178141 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:05.178196 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:05.178286 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 02:47:05.178554 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:05.179087 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:05.179922 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:05.180142 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:05.180228 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:05.180529 (+ 301us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 02:47:05.180936 (+ 407us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 02:47:05.181060 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:05.181115 (+ 55us) server_negotiation.cc:299] Negotiation successful
0925 02:47:05.181174 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260925 02:47:05.187184 1568 raft_consensus.cc:1261] T 05cc59745b4d4a1094a8266a389f85ee P ce3df7bf1e3d48248b12ec44683bd72b [term 1 FOLLOWER]: Refusing update from remote peer b051b8d15cc34d3091f2b4069b86ba68: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:47:05.187184 1830 raft_consensus.cc:1261] T 05cc59745b4d4a1094a8266a389f85ee P 039de77fe17441f583166d443559d043 [term 1 FOLLOWER]: Refusing update from remote peer b051b8d15cc34d3091f2b4069b86ba68: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:47:05.187848 1903 consensus_queue.cc:1035] T 05cc59745b4d4a1094a8266a389f85ee P b051b8d15cc34d3091f2b4069b86ba68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce3df7bf1e3d48248b12ec44683bd72b" member_type: VOTER last_known_addr { host: "127.13.157.129" port: 36797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:47:05.188023 1900 consensus_queue.cc:1035] T 05cc59745b4d4a1094a8266a389f85ee P b051b8d15cc34d3091f2b4069b86ba68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "039de77fe17441f583166d443559d043" member_type: VOTER last_known_addr { host: "127.13.157.131" port: 46157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:47:05.197127 1907 mvcc.cc:205] Tried to move back new op lower bound from 7333086925559734272 to 7333086925435645952. Current Snapshot: MvccSnapshot[applied={T|T < 7333086925559734272}]
I20260925 02:47:05.205606 1425 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60274:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 02:47:05.205765 1425 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60274:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 02:47:05.208254 1425 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 62fa485bead44f2ebb1a9f7e65c47d1a: Sending DeleteTablet for 3 replicas of tablet 05cc59745b4d4a1094a8266a389f85ee
I20260925 02:47:05.209059 1810 tablet_service.cc:1514] Processing DeleteTablet for tablet 05cc59745b4d4a1094a8266a389f85ee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:47:05 UTC) from {username='slave'} at 127.0.0.1:42822
I20260925 02:47:05.209038 1548 tablet_service.cc:1514] Processing DeleteTablet for tablet 05cc59745b4d4a1094a8266a389f85ee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:47:05 UTC) from {username='slave'} at 127.0.0.1:52244
I20260925 02:47:05.209580 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 1486
I20260925 02:47:05.209641 1679 tablet_service.cc:1514] Processing DeleteTablet for tablet 05cc59745b4d4a1094a8266a389f85ee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:47:05 UTC) from {username='slave'} at 127.0.0.1:39362
I20260925 02:47:05.210068 1917 tablet_replica.cc:330] T 05cc59745b4d4a1094a8266a389f85ee P 039de77fe17441f583166d443559d043: stopping tablet replica
I20260925 02:47:05.210361 1917 raft_consensus.cc:2229] T 05cc59745b4d4a1094a8266a389f85ee P 039de77fe17441f583166d443559d043 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 02:47:05.210664 1917 raft_consensus.cc:2258] T 05cc59745b4d4a1094a8266a389f85ee P 039de77fe17441f583166d443559d043 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:47:05.211117 1918 tablet_replica.cc:330] T 05cc59745b4d4a1094a8266a389f85ee P b051b8d15cc34d3091f2b4069b86ba68: stopping tablet replica
I20260925 02:47:05.211366 1918 raft_consensus.cc:2229] T 05cc59745b4d4a1094a8266a389f85ee P b051b8d15cc34d3091f2b4069b86ba68 [term 1 LEADER]: Raft consensus shutting down.
I20260925 02:47:05.211740 1918 raft_consensus.cc:2258] T 05cc59745b4d4a1094a8266a389f85ee P b051b8d15cc34d3091f2b4069b86ba68 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:47:05.212232 1917 ts_tablet_manager.cc:1905] T 05cc59745b4d4a1094a8266a389f85ee P 039de77fe17441f583166d443559d043: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:47:05.212934 1918 ts_tablet_manager.cc:1905] T 05cc59745b4d4a1094a8266a389f85ee P b051b8d15cc34d3091f2b4069b86ba68: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:47:05.214846 1917 ts_tablet_manager.cc:1918] T 05cc59745b4d4a1094a8266a389f85ee P 039de77fe17441f583166d443559d043: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 02:47:05.214959 1917 log.cc:1196] T 05cc59745b4d4a1094a8266a389f85ee P 039de77fe17441f583166d443559d043: Deleting WAL directory at /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790304178325258-13942-0/minicluster-data/ts-2/wal/wals/05cc59745b4d4a1094a8266a389f85ee
I20260925 02:47:05.215260 1917 ts_tablet_manager.cc:1939] T 05cc59745b4d4a1094a8266a389f85ee P 039de77fe17441f583166d443559d043: Deleting consensus metadata
I20260925 02:47:05.215523 1918 ts_tablet_manager.cc:1918] T 05cc59745b4d4a1094a8266a389f85ee P b051b8d15cc34d3091f2b4069b86ba68: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 02:47:05.215624 1918 log.cc:1196] T 05cc59745b4d4a1094a8266a389f85ee P b051b8d15cc34d3091f2b4069b86ba68: Deleting WAL directory at /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790304178325258-13942-0/minicluster-data/ts-1/wal/wals/05cc59745b4d4a1094a8266a389f85ee
I20260925 02:47:05.215891 1918 ts_tablet_manager.cc:1939] T 05cc59745b4d4a1094a8266a389f85ee P b051b8d15cc34d3091f2b4069b86ba68: Deleting consensus metadata
I20260925 02:47:05.216463 1411 catalog_manager.cc:4841] TS 039de77fe17441f583166d443559d043 (127.13.157.131:46157): tablet 05cc59745b4d4a1094a8266a389f85ee (table test-table [id=c2601cb498d64d4eb557b2531c92c085]) successfully deleted
I20260925 02:47:05.216933 1411 catalog_manager.cc:4841] TS b051b8d15cc34d3091f2b4069b86ba68 (127.13.157.130:34025): tablet 05cc59745b4d4a1094a8266a389f85ee (table test-table [id=c2601cb498d64d4eb557b2531c92c085]) successfully deleted
W20260925 02:47:05.220541 1413 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.157.129:36797 (error 108)
I20260925 02:47:05.220731 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 1617
I20260925 02:47:05.221992 1890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:05.221708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.129:36797 (local address 127.0.0.1:52250)
0925 02:47:05.221836 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0925 02:47:05.221908 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.129:36797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260925 02:47:05.222352 1413 catalog_manager.cc:4584] TS ce3df7bf1e3d48248b12ec44683bd72b (127.13.157.129:36797): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 05cc59745b4d4a1094a8266a389f85ee: Network error: Client connection negotiation failed: client connection to 127.13.157.129:36797: connect: Connection refused (error 111)
I20260925 02:47:05.225813 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 1748
I20260925 02:47:05.231175 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 1395
2026-09-25T02:47:05Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1003 ms)
[----------] 6 tests from EncryptionPolicyTest (9697 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790304178325258-13942-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 02:47:05 dist-test-slave-kznw krb5kdc[1922](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 02:47:05 dist-test-slave-kznw krb5kdc[1922](info): set up 2 sockets
Sep 25 02:47:05 dist-test-slave-kznw krb5kdc[1922](info): commencing operation
krb5kdc: starting...
W20260925 02:47:06.186353 13942 mini_kdc.cc:121] Time spent starting KDC: real 0.935s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790304178325258-13942-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790304178325258-13942-0/krb5kdc/test-user.keytab.
Sep 25 02:47:06 dist-test-slave-kznw krb5kdc[1922](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304426, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T02:47:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T02:47:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790304178325258-13942-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790304178325258-13942-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 02:47:06.303418 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790304178325258-13942-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790304178325258-13942-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790304178325258-13942-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35749
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790304178325258-13942-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35749
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:47:06.415027 1935 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:47:06.415274 1935 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:47:06.415331 1935 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:47:06.419310 1935 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:47:06.419363 1935 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:47:06.419389 1935 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:47:06.419418 1935 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:47:06.419449 1935 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:47:06.424587 1935 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790304178325258-13942-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790304178325258-13942-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35749
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790304178325258-13942-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35749
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1935
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790304178325258-13942-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:47:06.425585 1935 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:47:06.426594 1935 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:47:06.435859 1941 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:47:06.435858 1943 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:47:06.435877 1940 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:47:06.436332 1935 server_base.cc:1034] running on GCE node
I20260925 02:47:06.436983 1935 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:47:06.438311 1935 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:47:06.439482 1935 hybrid_clock.cc:648] HybridClock initialized: now 1790304426439456 us; error 33 us; skew 500 ppm
Sep 25 02:47:06 dist-test-slave-kznw krb5kdc[1922](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304426, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:47:06.445796 1935 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 02:47:06.446898 1935 webserver.cc:460] Webserver started at http://127.0.0.1:46063/ using document root <none> and password file <none>
I20260925 02:47:06.447602 1935 fs_manager.cc:359] Metadata directory not provided
I20260925 02:47:06.447685 1935 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:47:06.447918 1935 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:47:06.450685 1935 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790304178325258-13942-0/minicluster-data/master-0/data/instance:
uuid: "9fcdbe2ce30142ecb4441b1ca2347a08"
format_stamp: "Formatted at 2026-09-25 02:47:06 on dist-test-slave-kznw"
server_key: "dc511a2cdaa4b58f1ce4e8f88e6d2696"
server_key_iv: "c7e83117b0dac89b0b4603e446810ab7"
server_key_version: "encryptionkey@0"
I20260925 02:47:06.451416 1935 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790304178325258-13942-0/minicluster-data/master-0/wal/instance:
uuid: "9fcdbe2ce30142ecb4441b1ca2347a08"
format_stamp: "Formatted at 2026-09-25 02:47:06 on dist-test-slave-kznw"
server_key: "dc511a2cdaa4b58f1ce4e8f88e6d2696"
server_key_iv: "c7e83117b0dac89b0b4603e446810ab7"
server_key_version: "encryptionkey@0"
I20260925 02:47:06.456251 1935 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260925 02:47:06.459151 1950 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:47:06.460582 1935 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260925 02:47:06.460721 1935 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790304178325258-13942-0/minicluster-data/master-0/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790304178325258-13942-0/minicluster-data/master-0/wal
uuid: "9fcdbe2ce30142ecb4441b1ca2347a08"
format_stamp: "Formatted at 2026-09-25 02:47:06 on dist-test-slave-kznw"
server_key: "dc511a2cdaa4b58f1ce4e8f88e6d2696"
server_key_iv: "c7e83117b0dac89b0b4603e446810ab7"
server_key_version: "encryptionkey@0"
I20260925 02:47:06.460841 1935 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790304178325258-13942-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790304178325258-13942-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790304178325258-13942-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:47:06.478986 1935 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:47:06.482859 1935 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:47:06.483088 1935 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:47:06.491837 1935 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35749
I20260925 02:47:06.491954 2002 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35749 every 8 connection(s)
I20260925 02:47:06.492856 1935 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
I20260925 02:47:06.496310 2003 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:47:06.499512 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 1935
I20260925 02:47:06.499653 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790304178325258-13942-0/minicluster-data/master-0/wal/instance
I20260925 02:47:06.499963 13942 external_mini_cluster.cc:1428] Setting key f67b3006f08e9fa536cec2d2a4470cbc
I20260925 02:47:06.502336 2003 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9fcdbe2ce30142ecb4441b1ca2347a08: Bootstrap starting.
I20260925 02:47:06.504987 2003 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9fcdbe2ce30142ecb4441b1ca2347a08: Neither blocks nor log segments found. Creating new log.
I20260925 02:47:06.505801 2003 log.cc:824] T 00000000000000000000000000000000 P 9fcdbe2ce30142ecb4441b1ca2347a08: Log is configured to *not* fsync() on all Append() calls
I20260925 02:47:06.507695 2003 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9fcdbe2ce30142ecb4441b1ca2347a08: No bootstrap required, opened a new log
Sep 25 02:47:06 dist-test-slave-kznw krb5kdc[1922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304426, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260925 02:47:06.511555 2003 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9fcdbe2ce30142ecb4441b1ca2347a08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fcdbe2ce30142ecb4441b1ca2347a08" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35749 } }
I20260925 02:47:06.511793 2003 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9fcdbe2ce30142ecb4441b1ca2347a08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:47:06.511863 2003 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9fcdbe2ce30142ecb4441b1ca2347a08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9fcdbe2ce30142ecb4441b1ca2347a08, State: Initialized, Role: FOLLOWER
I20260925 02:47:06.512359 2003 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9fcdbe2ce30142ecb4441b1ca2347a08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fcdbe2ce30142ecb4441b1ca2347a08" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35749 } }
I20260925 02:47:06.512526 2003 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9fcdbe2ce30142ecb4441b1ca2347a08 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 02:47:06.512598 2003 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9fcdbe2ce30142ecb4441b1ca2347a08 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 02:47:06.512708 2003 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9fcdbe2ce30142ecb4441b1ca2347a08 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:47:06.513742 2003 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9fcdbe2ce30142ecb4441b1ca2347a08 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fcdbe2ce30142ecb4441b1ca2347a08" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35749 } }
I20260925 02:47:06.514063 2003 leader_election.cc:302] T 00000000000000000000000000000000 P 9fcdbe2ce30142ecb4441b1ca2347a08 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9fcdbe2ce30142ecb4441b1ca2347a08; no voters:
I20260925 02:47:06.514353 2003 leader_election.cc:288] T 00000000000000000000000000000000 P 9fcdbe2ce30142ecb4441b1ca2347a08 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 02:47:06.514441 2008 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9fcdbe2ce30142ecb4441b1ca2347a08 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:47:06.514643 2008 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9fcdbe2ce30142ecb4441b1ca2347a08 [term 1 LEADER]: Becoming Leader. State: Replica: 9fcdbe2ce30142ecb4441b1ca2347a08, State: Running, Role: LEADER
I20260925 02:47:06.514649 2006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:06.501321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40217 (local address 127.0.0.1:35749)
0925 02:47:06.501740 (+ 419us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:06.501760 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:06.501802 (+ 42us) server_negotiation.cc:407] Connection header received
0925 02:47:06.502597 (+ 795us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:06.502619 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:06.503067 (+ 448us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:06.503414 (+ 347us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:47:06.504052 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:06.505492 (+ 1440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:06.506407 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:06.506944 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:06.507108 (+ 164us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:06.509874 (+ 2766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:47:06.509879 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:47:06.509892 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:47:06.509920 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:47:06.511600 (+ 1680us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:47:06.512155 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:47:06.512161 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:47:06.512167 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:47:06.512286 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:47:06.512642 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:47:06.512644 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:47:06.512646 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:47:06.513067 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:47:06.513231 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:06.513537 (+ 306us) server_negotiation.cc:299] Negotiation successful
0925 02:47:06.513743 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":123,"threads_started":1}
I20260925 02:47:06.514984 2008 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9fcdbe2ce30142ecb4441b1ca2347a08 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fcdbe2ce30142ecb4441b1ca2347a08" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35749 } }
I20260925 02:47:06.515687 2003 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9fcdbe2ce30142ecb4441b1ca2347a08 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:47:06.516590 2010 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9fcdbe2ce30142ecb4441b1ca2347a08 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9fcdbe2ce30142ecb4441b1ca2347a08" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fcdbe2ce30142ecb4441b1ca2347a08" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35749 } } }
I20260925 02:47:06.516713 2010 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9fcdbe2ce30142ecb4441b1ca2347a08 [sys.catalog]: This master's current role is: LEADER
I20260925 02:47:06.517025 2009 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9fcdbe2ce30142ecb4441b1ca2347a08 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9fcdbe2ce30142ecb4441b1ca2347a08. Latest consensus state: current_term: 1 leader_uuid: "9fcdbe2ce30142ecb4441b1ca2347a08" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fcdbe2ce30142ecb4441b1ca2347a08" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35749 } } }
I20260925 02:47:06.517108 2009 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9fcdbe2ce30142ecb4441b1ca2347a08 [sys.catalog]: This master's current role is: LEADER
I20260925 02:47:06.517562 2017 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:47:06.517896 13942 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 02:47:06.520589 2017 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:47:06.526473 2017 catalog_manager.cc:1347] Generated new cluster ID: f2d41a1e2cc9454cb5179c10385a7ca5
I20260925 02:47:06.526546 2017 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 02:47:06.528088 2006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:06.520103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55015 (local address 127.0.0.1:35749)
0925 02:47:06.520203 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:06.520207 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:06.520538 (+ 331us) server_negotiation.cc:407] Connection header received
0925 02:47:06.520641 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:06.520644 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:06.520685 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:06.520746 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:47:06.521377 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:06.521873 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:06.522764 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:06.523135 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:06.523209 (+ 74us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:06.524837 (+ 1628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:47:06.524841 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:47:06.524844 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:47:06.524865 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:47:06.526375 (+ 1510us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:47:06.526871 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:47:06.526875 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:47:06.526876 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:47:06.526989 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:47:06.527301 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:47:06.527304 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:47:06.527306 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:47:06.527469 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:47:06.527600 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:06.527857 (+ 257us) server_negotiation.cc:299] Negotiation successful
0925 02:47:06.527943 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260925 02:47:06.559746 2017 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 02:47:06.560822 2017 catalog_manager.cc:1504] Loading token signing keys...
I20260925 02:47:06.564961 2017 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9fcdbe2ce30142ecb4441b1ca2347a08: Generated new TSK 0
I20260925 02:47:06.565793 2017 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 02:47:06.572876 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 1935
W20260925 02:47:06.578778 1920 connection.cc:535] client connection to 127.0.0.1:35749 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35749): Connection reset by peer (error 104)
2026-09-25T02:47:06Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1341 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790304178325258-13942-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 02:47:06 dist-test-slave-kznw krb5kdc[2034](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 02:47:06 dist-test-slave-kznw krb5kdc[2034](info): set up 2 sockets
Sep 25 02:47:06 dist-test-slave-kznw krb5kdc[2034](info): commencing operation
krb5kdc: starting...
W20260925 02:47:07.526722 13942 mini_kdc.cc:121] Time spent starting KDC: real 0.934s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790304178325258-13942-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790304178325258-13942-0/krb5kdc/test-user.keytab.
Sep 25 02:47:07 dist-test-slave-kznw krb5kdc[2034](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304427, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T02:47:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T02:47:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790304178325258-13942-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790304178325258-13942-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 02:47:07.645457 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790304178325258-13942-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790304178325258-13942-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790304178325258-13942-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39213
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790304178325258-13942-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39213
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:47:07.751236 2047 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:47:07.751492 2047 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:47:07.751533 2047 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:47:07.755434 2047 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:47:07.755496 2047 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:47:07.755513 2047 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:47:07.755529 2047 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:47:07.755542 2047 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:47:07.760730 2047 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790304178325258-13942-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790304178325258-13942-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39213
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790304178325258-13942-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39213
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2047
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790304178325258-13942-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:47:07.761698 2047 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:47:07.762624 2047 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:47:07.770733 2053 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:47:07.770772 2052 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:47:07.770730 2055 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:47:07.770994 2047 server_base.cc:1034] running on GCE node
I20260925 02:47:07.771653 2047 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:47:07.772773 2047 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:47:07.773933 2047 hybrid_clock.cc:648] HybridClock initialized: now 1790304427773917 us; error 30 us; skew 500 ppm
Sep 25 02:47:07 dist-test-slave-kznw krb5kdc[2034](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304427, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:47:07.776571 2047 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 02:47:07.777385 2047 webserver.cc:460] Webserver started at http://127.0.0.1:45967/ using document root <none> and password file <none>
I20260925 02:47:07.777957 2047 fs_manager.cc:359] Metadata directory not provided
I20260925 02:47:07.778028 2047 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:47:07.778215 2047 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:47:07.780257 2047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790304178325258-13942-0/minicluster-data/master-0/data/instance:
uuid: "7c22ce98c6804af99244537c4faf1371"
format_stamp: "Formatted at 2026-09-25 02:47:07 on dist-test-slave-kznw"
server_key: "407d5ea3a4093ae3ee31d57d72e91d29"
server_key_iv: "512f898e16cb42dbf7b5900ca94bfe09"
server_key_version: "encryptionkey@0"
I20260925 02:47:07.780799 2047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790304178325258-13942-0/minicluster-data/master-0/wal/instance:
uuid: "7c22ce98c6804af99244537c4faf1371"
format_stamp: "Formatted at 2026-09-25 02:47:07 on dist-test-slave-kznw"
server_key: "407d5ea3a4093ae3ee31d57d72e91d29"
server_key_iv: "512f898e16cb42dbf7b5900ca94bfe09"
server_key_version: "encryptionkey@0"
I20260925 02:47:07.784346 2047 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260925 02:47:07.786638 2062 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:47:07.787820 2047 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260925 02:47:07.787940 2047 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790304178325258-13942-0/minicluster-data/master-0/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790304178325258-13942-0/minicluster-data/master-0/wal
uuid: "7c22ce98c6804af99244537c4faf1371"
format_stamp: "Formatted at 2026-09-25 02:47:07 on dist-test-slave-kznw"
server_key: "407d5ea3a4093ae3ee31d57d72e91d29"
server_key_iv: "512f898e16cb42dbf7b5900ca94bfe09"
server_key_version: "encryptionkey@0"
I20260925 02:47:07.788029 2047 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790304178325258-13942-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790304178325258-13942-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790304178325258-13942-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:47:07.805693 2047 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:47:07.808772 2047 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:47:07.808950 2047 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:47:07.815757 2047 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39213
I20260925 02:47:07.815804 2114 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39213 every 8 connection(s)
I20260925 02:47:07.816694 2047 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
I20260925 02:47:07.819491 2115 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:47:07.821357 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 2047
I20260925 02:47:07.821482 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790304178325258-13942-0/minicluster-data/master-0/wal/instance
I20260925 02:47:07.821777 13942 external_mini_cluster.cc:1428] Setting key 6a5774898e2310c9c41bff5758c33703
I20260925 02:47:07.825441 2115 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c22ce98c6804af99244537c4faf1371: Bootstrap starting.
I20260925 02:47:07.827993 2115 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7c22ce98c6804af99244537c4faf1371: Neither blocks nor log segments found. Creating new log.
I20260925 02:47:07.828815 2115 log.cc:824] T 00000000000000000000000000000000 P 7c22ce98c6804af99244537c4faf1371: Log is configured to *not* fsync() on all Append() calls
I20260925 02:47:07.830793 2115 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c22ce98c6804af99244537c4faf1371: No bootstrap required, opened a new log
I20260925 02:47:07.830983 2118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:07.823185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50011 (local address 127.0.0.1:39213)
0925 02:47:07.823575 (+ 390us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:07.823590 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:07.823625 (+ 35us) server_negotiation.cc:407] Connection header received
0925 02:47:07.824415 (+ 790us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:07.824432 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:07.824809 (+ 377us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:07.825094 (+ 285us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:47:07.825453 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:07.826483 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:07.827693 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:07.828221 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:07.828353 (+ 132us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:07.829192 (+ 839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:47:07.829197 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:47:07.829210 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:47:07.829661 (+ 451us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:47:07.829714 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:47:07.829739 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:47:07.829839 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:07.830024 (+ 185us) server_negotiation.cc:299] Negotiation successful
0925 02:47:07.830158 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":244,"thread_start_us":98,"threads_started":1}
I20260925 02:47:07.833981 13942 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 02:47:07.834657 2115 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7c22ce98c6804af99244537c4faf1371 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c22ce98c6804af99244537c4faf1371" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39213 } }
I20260925 02:47:07.834887 2115 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7c22ce98c6804af99244537c4faf1371 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:47:07.834973 2115 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7c22ce98c6804af99244537c4faf1371 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c22ce98c6804af99244537c4faf1371, State: Initialized, Role: FOLLOWER
I20260925 02:47:07.835410 2115 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7c22ce98c6804af99244537c4faf1371 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c22ce98c6804af99244537c4faf1371" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39213 } }
I20260925 02:47:07.835546 2115 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7c22ce98c6804af99244537c4faf1371 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 02:47:07.835611 2115 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7c22ce98c6804af99244537c4faf1371 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 02:47:07.835711 2115 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7c22ce98c6804af99244537c4faf1371 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:47:07.836859 2115 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7c22ce98c6804af99244537c4faf1371 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c22ce98c6804af99244537c4faf1371" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39213 } }
I20260925 02:47:07.837198 2115 leader_election.cc:302] T 00000000000000000000000000000000 P 7c22ce98c6804af99244537c4faf1371 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7c22ce98c6804af99244537c4faf1371; no voters:
I20260925 02:47:07.837518 2115 leader_election.cc:288] T 00000000000000000000000000000000 P 7c22ce98c6804af99244537c4faf1371 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 02:47:07.837604 2125 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7c22ce98c6804af99244537c4faf1371 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:47:07.837818 2125 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7c22ce98c6804af99244537c4faf1371 [term 1 LEADER]: Becoming Leader. State: Replica: 7c22ce98c6804af99244537c4faf1371, State: Running, Role: LEADER
I20260925 02:47:07.838140 2125 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7c22ce98c6804af99244537c4faf1371 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c22ce98c6804af99244537c4faf1371" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39213 } }
I20260925 02:47:07.838773 2115 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7c22ce98c6804af99244537c4faf1371 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:47:07.840147 2126 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c22ce98c6804af99244537c4faf1371 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7c22ce98c6804af99244537c4faf1371" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c22ce98c6804af99244537c4faf1371" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39213 } } }
I20260925 02:47:07.840271 2126 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c22ce98c6804af99244537c4faf1371 [sys.catalog]: This master's current role is: LEADER
I20260925 02:47:07.840389 2118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:07.835935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34335 (local address 127.0.0.1:39213)
0925 02:47:07.836062 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:07.836066 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:07.836245 (+ 179us) server_negotiation.cc:407] Connection header received
0925 02:47:07.836362 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:07.836364 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:07.836402 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:07.836462 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:47:07.836738 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:07.837277 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:07.838399 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:07.838753 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:07.838852 (+ 99us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:07.839819 (+ 967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:47:07.839822 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:47:07.839825 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:47:07.839838 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:47:07.839865 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:47:07.839877 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:47:07.839967 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:07.840177 (+ 210us) server_negotiation.cc:299] Negotiation successful
0925 02:47:07.840210 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260925 02:47:07.840754 2127 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c22ce98c6804af99244537c4faf1371 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7c22ce98c6804af99244537c4faf1371. Latest consensus state: current_term: 1 leader_uuid: "7c22ce98c6804af99244537c4faf1371" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c22ce98c6804af99244537c4faf1371" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39213 } } }
I20260925 02:47:07.840845 2127 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c22ce98c6804af99244537c4faf1371 [sys.catalog]: This master's current role is: LEADER
W20260925 02:47:07.843695 2140 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7c22ce98c6804af99244537c4faf1371: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 02:47:07.843777 2140 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 02:47:07.843837 2137 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:47:07.844691 2137 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:47:07.850389 2137 catalog_manager.cc:1347] Generated new cluster ID: 2f9ff2cfc5ca48d28f63ff8eae39e217
I20260925 02:47:07.850461 2137 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 02:47:07.883307 2137 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 02:47:07.884631 2137 catalog_manager.cc:1504] Loading token signing keys...
I20260925 02:47:07.898016 2137 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7c22ce98c6804af99244537c4faf1371: Generated new TSK 0
I20260925 02:47:07.898808 2137 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 02:47:07.910188 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 2047
W20260925 02:47:07.915414 2032 connection.cc:535] client connection to 127.0.0.1:39213 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39213): Connection reset by peer (error 104)
2026-09-25T02:47:07Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1336 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790304178325258-13942-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 02:47:07 dist-test-slave-kznw krb5kdc[2146](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 02:47:07 dist-test-slave-kznw krb5kdc[2146](info): set up 2 sockets
Sep 25 02:47:07 dist-test-slave-kznw krb5kdc[2146](info): commencing operation
krb5kdc: starting...
W20260925 02:47:08.863240 13942 mini_kdc.cc:121] Time spent starting KDC: real 0.934s user 0.005s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790304178325258-13942-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790304178325258-13942-0/krb5kdc/test-user.keytab.
Sep 25 02:47:08 dist-test-slave-kznw krb5kdc[2146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304428, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T02:47:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T02:47:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790304178325258-13942-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790304178325258-13942-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 02:47:08.979725 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790304178325258-13942-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790304178325258-13942-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790304178325258-13942-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34137
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790304178325258-13942-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38905
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34137
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:47:09.087483 2159 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:47:09.087762 2159 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:47:09.087821 2159 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:47:09.091845 2159 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:47:09.091907 2159 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:47:09.091943 2159 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:47:09.091974 2159 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:47:09.091997 2159 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:47:09.097321 2159 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38905
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790304178325258-13942-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790304178325258-13942-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34137
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790304178325258-13942-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34137
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2159
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790304178325258-13942-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:47:09.098393 2159 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:47:09.099524 2159 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:47:09.108009 2167 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:47:09.108063 2164 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:47:09.108084 2165 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:47:09.108160 2159 server_base.cc:1034] running on GCE node
I20260925 02:47:09.108773 2159 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:47:09.109844 2159 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:47:09.110976 2159 hybrid_clock.cc:648] HybridClock initialized: now 1790304429110957 us; error 29 us; skew 500 ppm
Sep 25 02:47:09 dist-test-slave-kznw krb5kdc[2146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304429, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:47:09.113554 2159 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 02:47:09.114392 2159 webserver.cc:460] Webserver started at http://127.0.0.1:34607/ using document root <none> and password file <none>
I20260925 02:47:09.114897 2159 fs_manager.cc:359] Metadata directory not provided
I20260925 02:47:09.114997 2159 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:47:09.115198 2159 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:47:09.117163 2159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790304178325258-13942-0/minicluster-data/master-0/data/instance:
uuid: "dcd1bf9552ee4a5ca4f9e6b5687bb54b"
format_stamp: "Formatted at 2026-09-25 02:47:09 on dist-test-slave-kznw"
server_key: "4c5fc689bc9b4415e8a8ddfb91cf8679"
server_key_iv: "71c8a3137e30d421ce14f919dba2df50"
server_key_version: "encryptionkey@0"
I20260925 02:47:09.117686 2159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790304178325258-13942-0/minicluster-data/master-0/wal/instance:
uuid: "dcd1bf9552ee4a5ca4f9e6b5687bb54b"
format_stamp: "Formatted at 2026-09-25 02:47:09 on dist-test-slave-kznw"
server_key: "4c5fc689bc9b4415e8a8ddfb91cf8679"
server_key_iv: "71c8a3137e30d421ce14f919dba2df50"
server_key_version: "encryptionkey@0"
I20260925 02:47:09.121261 2159 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.005s
I20260925 02:47:09.123538 2174 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:47:09.124711 2159 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.002s
I20260925 02:47:09.124816 2159 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790304178325258-13942-0/minicluster-data/master-0/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790304178325258-13942-0/minicluster-data/master-0/wal
uuid: "dcd1bf9552ee4a5ca4f9e6b5687bb54b"
format_stamp: "Formatted at 2026-09-25 02:47:09 on dist-test-slave-kznw"
server_key: "4c5fc689bc9b4415e8a8ddfb91cf8679"
server_key_iv: "71c8a3137e30d421ce14f919dba2df50"
server_key_version: "encryptionkey@0"
I20260925 02:47:09.124903 2159 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790304178325258-13942-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790304178325258-13942-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790304178325258-13942-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:47:09.129240 2159 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:47:09.131853 2159 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:47:09.132004 2159 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:47:09.138545 2159 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34137
I20260925 02:47:09.138581 2226 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34137 every 8 connection(s)
I20260925 02:47:09.139424 2159 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
I20260925 02:47:09.142117 2227 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:47:09.145259 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 2159
I20260925 02:47:09.145406 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790304178325258-13942-0/minicluster-data/master-0/wal/instance
I20260925 02:47:09.145761 13942 external_mini_cluster.cc:1428] Setting key 6675eca396b16e3fc282f7d1bbe5ac53
I20260925 02:47:09.148250 2227 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dcd1bf9552ee4a5ca4f9e6b5687bb54b: Bootstrap starting.
I20260925 02:47:09.150858 2227 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dcd1bf9552ee4a5ca4f9e6b5687bb54b: Neither blocks nor log segments found. Creating new log.
I20260925 02:47:09.151090 2230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:09.147180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37683 (local address 127.0.0.1:34137)
0925 02:47:09.147561 (+ 381us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:09.147576 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:09.147610 (+ 34us) server_negotiation.cc:407] Connection header received
0925 02:47:09.148222 (+ 612us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:09.148239 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:09.148539 (+ 300us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:09.148821 (+ 282us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:47:09.149693 (+ 872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:47:09.149699 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:47:09.149714 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:47:09.149728 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:47:09.149788 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:47:09.149809 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:47:09.149929 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:09.150048 (+ 119us) server_negotiation.cc:299] Negotiation successful
0925 02:47:09.150190 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":247,"thread_start_us":103,"threads_started":1}
I20260925 02:47:09.151832 2227 log.cc:824] T 00000000000000000000000000000000 P dcd1bf9552ee4a5ca4f9e6b5687bb54b: Log is configured to *not* fsync() on all Append() calls
I20260925 02:47:09.153753 2227 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dcd1bf9552ee4a5ca4f9e6b5687bb54b: No bootstrap required, opened a new log
I20260925 02:47:09.153851 13942 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 02:47:09.157337 2230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:09.155827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47855 (local address 127.0.0.1:34137)
0925 02:47:09.155931 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:09.155935 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:09.156122 (+ 187us) server_negotiation.cc:407] Connection header received
0925 02:47:09.156224 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:09.156227 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:09.156261 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:09.156322 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:47:09.156985 (+ 663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:47:09.156988 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:47:09.156991 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:47:09.157002 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:47:09.157030 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:47:09.157040 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:47:09.157109 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:09.157183 (+ 74us) server_negotiation.cc:299] Negotiation successful
0925 02:47:09.157221 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260925 02:47:09.157519 2227 raft_consensus.cc:348] T 00000000000000000000000000000000 P dcd1bf9552ee4a5ca4f9e6b5687bb54b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcd1bf9552ee4a5ca4f9e6b5687bb54b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34137 } }
I20260925 02:47:09.157778 2227 raft_consensus.cc:374] T 00000000000000000000000000000000 P dcd1bf9552ee4a5ca4f9e6b5687bb54b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:47:09.157833 2227 raft_consensus.cc:729] T 00000000000000000000000000000000 P dcd1bf9552ee4a5ca4f9e6b5687bb54b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dcd1bf9552ee4a5ca4f9e6b5687bb54b, State: Initialized, Role: FOLLOWER
I20260925 02:47:09.158279 2227 consensus_queue.cc:260] T 00000000000000000000000000000000 P dcd1bf9552ee4a5ca4f9e6b5687bb54b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcd1bf9552ee4a5ca4f9e6b5687bb54b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34137 } }
I20260925 02:47:09.158434 2227 raft_consensus.cc:388] T 00000000000000000000000000000000 P dcd1bf9552ee4a5ca4f9e6b5687bb54b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 02:47:09.158491 2227 raft_consensus.cc:482] T 00000000000000000000000000000000 P dcd1bf9552ee4a5ca4f9e6b5687bb54b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 02:47:09.158586 2227 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dcd1bf9552ee4a5ca4f9e6b5687bb54b [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:47:09.159565 2227 raft_consensus.cc:504] T 00000000000000000000000000000000 P dcd1bf9552ee4a5ca4f9e6b5687bb54b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcd1bf9552ee4a5ca4f9e6b5687bb54b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34137 } }
I20260925 02:47:09.159881 2227 leader_election.cc:302] T 00000000000000000000000000000000 P dcd1bf9552ee4a5ca4f9e6b5687bb54b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dcd1bf9552ee4a5ca4f9e6b5687bb54b; no voters:
I20260925 02:47:09.160161 2227 leader_election.cc:288] T 00000000000000000000000000000000 P dcd1bf9552ee4a5ca4f9e6b5687bb54b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 02:47:09.160285 2237 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dcd1bf9552ee4a5ca4f9e6b5687bb54b [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:47:09.160480 2237 raft_consensus.cc:686] T 00000000000000000000000000000000 P dcd1bf9552ee4a5ca4f9e6b5687bb54b [term 1 LEADER]: Becoming Leader. State: Replica: dcd1bf9552ee4a5ca4f9e6b5687bb54b, State: Running, Role: LEADER
I20260925 02:47:09.160815 2237 consensus_queue.cc:237] T 00000000000000000000000000000000 P dcd1bf9552ee4a5ca4f9e6b5687bb54b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcd1bf9552ee4a5ca4f9e6b5687bb54b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34137 } }
I20260925 02:47:09.161299 2227 sys_catalog.cc:564] T 00000000000000000000000000000000 P dcd1bf9552ee4a5ca4f9e6b5687bb54b [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:47:09.162403 2238 sys_catalog.cc:455] T 00000000000000000000000000000000 P dcd1bf9552ee4a5ca4f9e6b5687bb54b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dcd1bf9552ee4a5ca4f9e6b5687bb54b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcd1bf9552ee4a5ca4f9e6b5687bb54b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34137 } } }
I20260925 02:47:09.162452 2239 sys_catalog.cc:455] T 00000000000000000000000000000000 P dcd1bf9552ee4a5ca4f9e6b5687bb54b [sys.catalog]: SysCatalogTable state changed. Reason: New leader dcd1bf9552ee4a5ca4f9e6b5687bb54b. Latest consensus state: current_term: 1 leader_uuid: "dcd1bf9552ee4a5ca4f9e6b5687bb54b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcd1bf9552ee4a5ca4f9e6b5687bb54b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34137 } } }
I20260925 02:47:09.162528 2238 sys_catalog.cc:458] T 00000000000000000000000000000000 P dcd1bf9552ee4a5ca4f9e6b5687bb54b [sys.catalog]: This master's current role is: LEADER
I20260925 02:47:09.162546 2239 sys_catalog.cc:458] T 00000000000000000000000000000000 P dcd1bf9552ee4a5ca4f9e6b5687bb54b [sys.catalog]: This master's current role is: LEADER
I20260925 02:47:09.162873 2246 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:47:09.165946 2246 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:47:09.171350 2246 catalog_manager.cc:1347] Generated new cluster ID: 8af7c88e20fc47ce8a93562adcf81822
I20260925 02:47:09.171401 2246 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 02:47:09.186226 2246 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 02:47:09.186525 2246 catalog_manager.cc:1504] Loading token signing keys...
I20260925 02:47:09.194023 2246 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dcd1bf9552ee4a5ca4f9e6b5687bb54b: Generated new TSK 0
I20260925 02:47:09.194661 2246 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 02:47:09.201051 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 2159
2026-09-25T02:47:09Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1290 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790304178325258-13942-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 02:47:09 dist-test-slave-kznw krb5kdc[2258](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 02:47:09 dist-test-slave-kznw krb5kdc[2258](info): set up 2 sockets
Sep 25 02:47:09 dist-test-slave-kznw krb5kdc[2258](info): commencing operation
krb5kdc: starting...
W20260925 02:47:10.162508 13942 mini_kdc.cc:121] Time spent starting KDC: real 0.942s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790304178325258-13942-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790304178325258-13942-0/krb5kdc/test-user.keytab.
Sep 25 02:47:10 dist-test-slave-kznw krb5kdc[2258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304430, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T02:47:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T02:47:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790304178325258-13942-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790304178325258-13942-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 02:47:10.275830 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790304178325258-13942-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790304178325258-13942-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790304178325258-13942-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44843
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790304178325258-13942-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44843
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:47:10.387171 2271 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:47:10.387463 2271 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:47:10.387602 2271 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:47:10.391688 2271 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:47:10.391749 2271 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:47:10.391768 2271 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:47:10.391784 2271 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:47:10.391801 2271 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:47:10.397253 2271 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790304178325258-13942-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790304178325258-13942-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44843
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790304178325258-13942-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44843
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2271
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790304178325258-13942-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:47:10.398420 2271 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:47:10.399555 2271 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:47:10.408159 2276 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:47:10.408142 2279 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:47:10.408159 2277 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:47:10.408418 2271 server_base.cc:1034] running on GCE node
I20260925 02:47:10.408979 2271 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:47:10.410097 2271 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:47:10.411278 2271 hybrid_clock.cc:648] HybridClock initialized: now 1790304430411250 us; error 37 us; skew 500 ppm
Sep 25 02:47:10 dist-test-slave-kznw krb5kdc[2258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304430, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:47:10.414153 2271 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 02:47:10.415163 2271 webserver.cc:460] Webserver started at http://127.0.0.1:44681/ using document root <none> and password file <none>
I20260925 02:47:10.415704 2271 fs_manager.cc:359] Metadata directory not provided
I20260925 02:47:10.415764 2271 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:47:10.415982 2271 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:47:10.418045 2271 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790304178325258-13942-0/minicluster-data/master-0/data/instance:
uuid: "4f3e0fcb23594d38bd65b443f8092f0f"
format_stamp: "Formatted at 2026-09-25 02:47:10 on dist-test-slave-kznw"
server_key: "b8dd0e1f231eef50d68d94cd9fa28ef3"
server_key_iv: "afe3def62c982c65b85c60a6ffa56baa"
server_key_version: "encryptionkey@0"
I20260925 02:47:10.418613 2271 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790304178325258-13942-0/minicluster-data/master-0/wal/instance:
uuid: "4f3e0fcb23594d38bd65b443f8092f0f"
format_stamp: "Formatted at 2026-09-25 02:47:10 on dist-test-slave-kznw"
server_key: "b8dd0e1f231eef50d68d94cd9fa28ef3"
server_key_iv: "afe3def62c982c65b85c60a6ffa56baa"
server_key_version: "encryptionkey@0"
I20260925 02:47:10.422674 2271 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260925 02:47:10.425225 2286 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:47:10.426582 2271 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260925 02:47:10.426708 2271 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790304178325258-13942-0/minicluster-data/master-0/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790304178325258-13942-0/minicluster-data/master-0/wal
uuid: "4f3e0fcb23594d38bd65b443f8092f0f"
format_stamp: "Formatted at 2026-09-25 02:47:10 on dist-test-slave-kznw"
server_key: "b8dd0e1f231eef50d68d94cd9fa28ef3"
server_key_iv: "afe3def62c982c65b85c60a6ffa56baa"
server_key_version: "encryptionkey@0"
I20260925 02:47:10.426813 2271 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790304178325258-13942-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790304178325258-13942-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790304178325258-13942-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:47:10.441406 2271 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:47:10.444633 2271 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:47:10.444808 2271 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:47:10.451792 2271 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44843
I20260925 02:47:10.451879 2338 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44843 every 8 connection(s)
I20260925 02:47:10.452783 2271 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
I20260925 02:47:10.455668 2339 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:47:10.461244 2339 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f3e0fcb23594d38bd65b443f8092f0f: Bootstrap starting.
I20260925 02:47:10.462067 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 2271
I20260925 02:47:10.462195 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790304178325258-13942-0/minicluster-data/master-0/wal/instance
I20260925 02:47:10.462462 13942 external_mini_cluster.cc:1428] Setting key 92f724350934c57afca7bee7b588a4d9
I20260925 02:47:10.463945 2339 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4f3e0fcb23594d38bd65b443f8092f0f: Neither blocks nor log segments found. Creating new log.
I20260925 02:47:10.464655 2339 log.cc:824] T 00000000000000000000000000000000 P 4f3e0fcb23594d38bd65b443f8092f0f: Log is configured to *not* fsync() on all Append() calls
I20260925 02:47:10.466806 2339 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f3e0fcb23594d38bd65b443f8092f0f: No bootstrap required, opened a new log
I20260925 02:47:10.470968 2339 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4f3e0fcb23594d38bd65b443f8092f0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f3e0fcb23594d38bd65b443f8092f0f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44843 } }
I20260925 02:47:10.471182 2339 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4f3e0fcb23594d38bd65b443f8092f0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
Sep 25 02:47:10 dist-test-slave-kznw krb5kdc[2258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304430, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260925 02:47:10.471239 2339 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4f3e0fcb23594d38bd65b443f8092f0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f3e0fcb23594d38bd65b443f8092f0f, State: Initialized, Role: FOLLOWER
I20260925 02:47:10.471699 2339 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4f3e0fcb23594d38bd65b443f8092f0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f3e0fcb23594d38bd65b443f8092f0f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44843 } }
I20260925 02:47:10.471823 2339 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4f3e0fcb23594d38bd65b443f8092f0f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 02:47:10.471882 2339 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4f3e0fcb23594d38bd65b443f8092f0f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 02:47:10.471972 2339 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4f3e0fcb23594d38bd65b443f8092f0f [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:47:10.472924 2339 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4f3e0fcb23594d38bd65b443f8092f0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f3e0fcb23594d38bd65b443f8092f0f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44843 } }
I20260925 02:47:10.473240 2339 leader_election.cc:302] T 00000000000000000000000000000000 P 4f3e0fcb23594d38bd65b443f8092f0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4f3e0fcb23594d38bd65b443f8092f0f; no voters:
I20260925 02:47:10.473523 2339 leader_election.cc:288] T 00000000000000000000000000000000 P 4f3e0fcb23594d38bd65b443f8092f0f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 02:47:10.473601 2344 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4f3e0fcb23594d38bd65b443f8092f0f [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:47:10.473788 2344 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4f3e0fcb23594d38bd65b443f8092f0f [term 1 LEADER]: Becoming Leader. State: Replica: 4f3e0fcb23594d38bd65b443f8092f0f, State: Running, Role: LEADER
I20260925 02:47:10.474143 2344 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4f3e0fcb23594d38bd65b443f8092f0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f3e0fcb23594d38bd65b443f8092f0f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44843 } }
I20260925 02:47:10.474815 2339 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4f3e0fcb23594d38bd65b443f8092f0f [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:47:10.475849 2346 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f3e0fcb23594d38bd65b443f8092f0f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4f3e0fcb23594d38bd65b443f8092f0f. Latest consensus state: current_term: 1 leader_uuid: "4f3e0fcb23594d38bd65b443f8092f0f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f3e0fcb23594d38bd65b443f8092f0f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44843 } } }
I20260925 02:47:10.475979 2346 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f3e0fcb23594d38bd65b443f8092f0f [sys.catalog]: This master's current role is: LEADER
I20260925 02:47:10.476315 2345 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f3e0fcb23594d38bd65b443f8092f0f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4f3e0fcb23594d38bd65b443f8092f0f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f3e0fcb23594d38bd65b443f8092f0f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44843 } } }
I20260925 02:47:10.476456 2345 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f3e0fcb23594d38bd65b443f8092f0f [sys.catalog]: This master's current role is: LEADER
I20260925 02:47:10.477370 2342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:10.463553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51175 (local address 127.0.0.1:44843)
0925 02:47:10.463943 (+ 390us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:10.463959 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:10.463996 (+ 37us) server_negotiation.cc:407] Connection header received
0925 02:47:10.464655 (+ 659us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:10.464671 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:10.465052 (+ 381us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:10.465438 (+ 386us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:47:10.466229 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:10.467300 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:10.468472 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:10.468942 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:10.469066 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:10.471736 (+ 2670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:47:10.471740 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:47:10.471752 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:47:10.471778 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:47:10.473540 (+ 1762us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:47:10.474047 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:47:10.474053 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:47:10.474058 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:47:10.474194 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:47:10.474491 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:47:10.474494 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:47:10.474496 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:47:10.474871 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:47:10.475073 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:10.475808 (+ 735us) server_negotiation.cc:299] Negotiation successful
0925 02:47:10.476102 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":254,"thread_start_us":100,"threads_started":1}
W20260925 02:47:10.479813 2359 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4f3e0fcb23594d38bd65b443f8092f0f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 02:47:10.479890 2359 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 02:47:10.479959 2353 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:47:10.480113 13942 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 02:47:10.480701 2353 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:47:10.486246 2353 catalog_manager.cc:1347] Generated new cluster ID: 188fa1694a8e4f4ca1f33cd11c3e594b
I20260925 02:47:10.486310 2353 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 02:47:10.489384 2342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:10.482000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60983 (local address 127.0.0.1:44843)
0925 02:47:10.482110 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:10.482115 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:10.482312 (+ 197us) server_negotiation.cc:407] Connection header received
0925 02:47:10.482411 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:10.482414 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:10.482456 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:10.482522 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:47:10.483167 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:10.483675 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:10.484585 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:10.484857 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:10.484960 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:10.486553 (+ 1593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:47:10.486563 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:47:10.486565 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:47:10.486589 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:47:10.487912 (+ 1323us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:47:10.488288 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:47:10.488291 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:47:10.488293 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:47:10.488365 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:47:10.488612 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:47:10.488615 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:47:10.488617 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:47:10.488779 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:47:10.488884 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:10.489120 (+ 236us) server_negotiation.cc:299] Negotiation successful
0925 02:47:10.489226 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260925 02:47:10.504051 2353 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 02:47:10.505018 2353 catalog_manager.cc:1504] Loading token signing keys...
I20260925 02:47:10.516388 2353 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4f3e0fcb23594d38bd65b443f8092f0f: Generated new TSK 0
I20260925 02:47:10.517169 2353 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 02:47:10.524169 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 2271
2026-09-25T02:47:10Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1323 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790304178325258-13942-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 02:47:10 dist-test-slave-kznw krb5kdc[2370](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 02:47:10 dist-test-slave-kznw krb5kdc[2370](info): set up 2 sockets
Sep 25 02:47:10 dist-test-slave-kznw krb5kdc[2370](info): commencing operation
krb5kdc: starting...
W20260925 02:47:11.482456 13942 mini_kdc.cc:121] Time spent starting KDC: real 0.939s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790304178325258-13942-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790304178325258-13942-0/krb5kdc/test-user.keytab.
Sep 25 02:47:11 dist-test-slave-kznw krb5kdc[2370](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304431, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T02:47:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T02:47:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790304178325258-13942-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790304178325258-13942-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 02:47:11.599825 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790304178325258-13942-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790304178325258-13942-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790304178325258-13942-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46595
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790304178325258-13942-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46595
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:47:11.708227 2383 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:47:11.708494 2383 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:47:11.708542 2383 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:47:11.712491 2383 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:47:11.712546 2383 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:47:11.712564 2383 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:47:11.712580 2383 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:47:11.712594 2383 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:47:11.717875 2383 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790304178325258-13942-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790304178325258-13942-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46595
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790304178325258-13942-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46595
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2383
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790304178325258-13942-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:47:11.718883 2383 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:47:11.719986 2383 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:47:11.728504 2391 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:47:11.728495 2389 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:47:11.728552 2383 server_base.cc:1034] running on GCE node
W20260925 02:47:11.728500 2388 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:47:11.729382 2383 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:47:11.730481 2383 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:47:11.731647 2383 hybrid_clock.cc:648] HybridClock initialized: now 1790304431731632 us; error 26 us; skew 500 ppm
Sep 25 02:47:11 dist-test-slave-kznw krb5kdc[2370](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304431, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:47:11.738277 2383 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 02:47:11.739398 2383 webserver.cc:460] Webserver started at http://127.0.0.1:36803/ using document root <none> and password file <none>
I20260925 02:47:11.739923 2383 fs_manager.cc:359] Metadata directory not provided
I20260925 02:47:11.739982 2383 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:47:11.740151 2383 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:47:11.742219 2383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790304178325258-13942-0/minicluster-data/master-0/data/instance:
uuid: "bd39e58a0da14637b920878c03bb7606"
format_stamp: "Formatted at 2026-09-25 02:47:11 on dist-test-slave-kznw"
server_key: "7b60d044703248443b24aba1cf0977cb"
server_key_iv: "51ad9e762e4a53d304e77d5e6bfb91e0"
server_key_version: "encryptionkey@0"
I20260925 02:47:11.742739 2383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790304178325258-13942-0/minicluster-data/master-0/wal/instance:
uuid: "bd39e58a0da14637b920878c03bb7606"
format_stamp: "Formatted at 2026-09-25 02:47:11 on dist-test-slave-kznw"
server_key: "7b60d044703248443b24aba1cf0977cb"
server_key_iv: "51ad9e762e4a53d304e77d5e6bfb91e0"
server_key_version: "encryptionkey@0"
I20260925 02:47:11.746448 2383 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260925 02:47:11.748862 2398 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:47:11.750123 2383 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260925 02:47:11.750247 2383 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790304178325258-13942-0/minicluster-data/master-0/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790304178325258-13942-0/minicluster-data/master-0/wal
uuid: "bd39e58a0da14637b920878c03bb7606"
format_stamp: "Formatted at 2026-09-25 02:47:11 on dist-test-slave-kznw"
server_key: "7b60d044703248443b24aba1cf0977cb"
server_key_iv: "51ad9e762e4a53d304e77d5e6bfb91e0"
server_key_version: "encryptionkey@0"
I20260925 02:47:11.750355 2383 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790304178325258-13942-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790304178325258-13942-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790304178325258-13942-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:47:11.762882 2383 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:47:11.766165 2383 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:47:11.766377 2383 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:47:11.773751 2383 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46595
I20260925 02:47:11.773782 2450 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46595 every 8 connection(s)
I20260925 02:47:11.774708 2383 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
I20260925 02:47:11.775480 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 2383
I20260925 02:47:11.775630 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790304178325258-13942-0/minicluster-data/master-0/wal/instance
I20260925 02:47:11.775939 13942 external_mini_cluster.cc:1428] Setting key 514afa6e5a18626e110e818be5235de1
I20260925 02:47:11.777971 2451 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:47:11.784164 2451 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd39e58a0da14637b920878c03bb7606: Bootstrap starting.
I20260925 02:47:11.784778 2454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:11.777292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36593 (local address 127.0.0.1:46595)
0925 02:47:11.777718 (+ 426us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:11.777738 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:11.777775 (+ 37us) server_negotiation.cc:407] Connection header received
0925 02:47:11.778451 (+ 676us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:11.778470 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:11.778814 (+ 344us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:11.779149 (+ 335us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:47:11.779461 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:11.780438 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:11.781612 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:11.782073 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:11.782184 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:11.783087 (+ 903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:47:11.783092 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:47:11.783105 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:47:11.783481 (+ 376us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:47:11.783534 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:47:11.783557 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:47:11.783630 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:11.783783 (+ 153us) server_negotiation.cc:299] Negotiation successful
0925 02:47:11.783988 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":290,"thread_start_us":167,"threads_started":1}
I20260925 02:47:11.786473 2451 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bd39e58a0da14637b920878c03bb7606: Neither blocks nor log segments found. Creating new log.
I20260925 02:47:11.787271 2451 log.cc:824] T 00000000000000000000000000000000 P bd39e58a0da14637b920878c03bb7606: Log is configured to *not* fsync() on all Append() calls
I20260925 02:47:11.787510 13942 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 02:47:11.789358 2451 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd39e58a0da14637b920878c03bb7606: No bootstrap required, opened a new log
I20260925 02:47:11.793308 2454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:11.789557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53235 (local address 127.0.0.1:46595)
0925 02:47:11.789667 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:11.789670 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:11.789786 (+ 116us) server_negotiation.cc:407] Connection header received
0925 02:47:11.789897 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:11.789899 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:11.789943 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:11.790004 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:47:11.790286 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:11.790791 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:11.791909 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:11.792125 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:11.792201 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:11.792924 (+ 723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:47:11.792926 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:47:11.792928 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:47:11.792939 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:47:11.792963 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:47:11.792973 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:47:11.793024 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:11.793137 (+ 113us) server_negotiation.cc:299] Negotiation successful
0925 02:47:11.793181 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260925 02:47:11.793812 2451 raft_consensus.cc:348] T 00000000000000000000000000000000 P bd39e58a0da14637b920878c03bb7606 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd39e58a0da14637b920878c03bb7606" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46595 } }
I20260925 02:47:11.794044 2451 raft_consensus.cc:374] T 00000000000000000000000000000000 P bd39e58a0da14637b920878c03bb7606 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:47:11.794100 2451 raft_consensus.cc:729] T 00000000000000000000000000000000 P bd39e58a0da14637b920878c03bb7606 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd39e58a0da14637b920878c03bb7606, State: Initialized, Role: FOLLOWER
I20260925 02:47:11.794500 2451 consensus_queue.cc:260] T 00000000000000000000000000000000 P bd39e58a0da14637b920878c03bb7606 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd39e58a0da14637b920878c03bb7606" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46595 } }
I20260925 02:47:11.794623 2451 raft_consensus.cc:388] T 00000000000000000000000000000000 P bd39e58a0da14637b920878c03bb7606 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 02:47:11.794687 2451 raft_consensus.cc:482] T 00000000000000000000000000000000 P bd39e58a0da14637b920878c03bb7606 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 02:47:11.794780 2451 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bd39e58a0da14637b920878c03bb7606 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:47:11.795667 2451 raft_consensus.cc:504] T 00000000000000000000000000000000 P bd39e58a0da14637b920878c03bb7606 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd39e58a0da14637b920878c03bb7606" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46595 } }
I20260925 02:47:11.795984 2451 leader_election.cc:302] T 00000000000000000000000000000000 P bd39e58a0da14637b920878c03bb7606 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bd39e58a0da14637b920878c03bb7606; no voters:
I20260925 02:47:11.796260 2451 leader_election.cc:288] T 00000000000000000000000000000000 P bd39e58a0da14637b920878c03bb7606 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 02:47:11.796363 2461 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bd39e58a0da14637b920878c03bb7606 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:47:11.796595 2461 raft_consensus.cc:686] T 00000000000000000000000000000000 P bd39e58a0da14637b920878c03bb7606 [term 1 LEADER]: Becoming Leader. State: Replica: bd39e58a0da14637b920878c03bb7606, State: Running, Role: LEADER
I20260925 02:47:11.796984 2461 consensus_queue.cc:237] T 00000000000000000000000000000000 P bd39e58a0da14637b920878c03bb7606 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd39e58a0da14637b920878c03bb7606" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46595 } }
I20260925 02:47:11.797523 2451 sys_catalog.cc:564] T 00000000000000000000000000000000 P bd39e58a0da14637b920878c03bb7606 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:47:11.798609 2462 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd39e58a0da14637b920878c03bb7606 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bd39e58a0da14637b920878c03bb7606" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd39e58a0da14637b920878c03bb7606" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46595 } } }
I20260925 02:47:11.798740 2462 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd39e58a0da14637b920878c03bb7606 [sys.catalog]: This master's current role is: LEADER
I20260925 02:47:11.799084 2463 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd39e58a0da14637b920878c03bb7606 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bd39e58a0da14637b920878c03bb7606. Latest consensus state: current_term: 1 leader_uuid: "bd39e58a0da14637b920878c03bb7606" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd39e58a0da14637b920878c03bb7606" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46595 } } }
I20260925 02:47:11.799196 2470 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:47:11.799263 2463 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd39e58a0da14637b920878c03bb7606 [sys.catalog]: This master's current role is: LEADER
I20260925 02:47:11.802007 2470 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:47:11.807304 2470 catalog_manager.cc:1347] Generated new cluster ID: 8104e1273c8c4cfdb28c57d57b2f700a
I20260925 02:47:11.807391 2470 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 02:47:11.828658 2470 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 02:47:11.830003 2470 catalog_manager.cc:1504] Loading token signing keys...
I20260925 02:47:11.846246 2470 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bd39e58a0da14637b920878c03bb7606: Generated new TSK 0
I20260925 02:47:11.847168 2470 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 02:47:11.862311 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 2383
2026-09-25T02:47:11Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1338 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790304178325258-13942-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 02:47:11 dist-test-slave-kznw krb5kdc[2482](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 02:47:11 dist-test-slave-kznw krb5kdc[2482](info): set up 2 sockets
Sep 25 02:47:11 dist-test-slave-kznw krb5kdc[2482](info): commencing operation
krb5kdc: starting...
W20260925 02:47:12.818491 13942 mini_kdc.cc:121] Time spent starting KDC: real 0.936s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790304178325258-13942-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790304178325258-13942-0/krb5kdc/test-user.keytab.
Sep 25 02:47:12 dist-test-slave-kznw krb5kdc[2482](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304432, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T02:47:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T02:47:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790304178325258-13942-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790304178325258-13942-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 02:47:12.939687 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790304178325258-13942-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790304178325258-13942-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790304178325258-13942-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35117
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790304178325258-13942-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35117
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:47:13.047042 2495 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:47:13.047291 2495 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:47:13.047333 2495 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:47:13.051235 2495 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:47:13.051290 2495 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:47:13.051307 2495 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:47:13.051322 2495 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:47:13.051337 2495 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:47:13.056409 2495 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790304178325258-13942-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790304178325258-13942-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35117
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790304178325258-13942-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35117
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2495
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790304178325258-13942-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:47:13.057394 2495 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:47:13.058461 2495 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:47:13.066648 2503 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:47:13.066648 2500 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:47:13.066811 2495 server_base.cc:1034] running on GCE node
W20260925 02:47:13.066665 2501 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:47:13.067456 2495 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:47:13.068535 2495 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:47:13.069703 2495 hybrid_clock.cc:648] HybridClock initialized: now 1790304433069687 us; error 26 us; skew 500 ppm
Sep 25 02:47:13 dist-test-slave-kznw krb5kdc[2482](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304433, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:47:13.072333 2495 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 02:47:13.073158 2495 webserver.cc:460] Webserver started at http://127.0.0.1:43051/ using document root <none> and password file <none>
I20260925 02:47:13.073683 2495 fs_manager.cc:359] Metadata directory not provided
I20260925 02:47:13.073750 2495 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:47:13.073948 2495 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:47:13.075982 2495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790304178325258-13942-0/minicluster-data/master-0/data/instance:
uuid: "648825c0b9ab4f78a8aeb52410287517"
format_stamp: "Formatted at 2026-09-25 02:47:13 on dist-test-slave-kznw"
server_key: "57f1a3132a5ba766508bb53eaa7db16e"
server_key_iv: "bb325b08e8d62cc1657b3769aa1a59ef"
server_key_version: "encryptionkey@0"
I20260925 02:47:13.076494 2495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790304178325258-13942-0/minicluster-data/master-0/wal/instance:
uuid: "648825c0b9ab4f78a8aeb52410287517"
format_stamp: "Formatted at 2026-09-25 02:47:13 on dist-test-slave-kznw"
server_key: "57f1a3132a5ba766508bb53eaa7db16e"
server_key_iv: "bb325b08e8d62cc1657b3769aa1a59ef"
server_key_version: "encryptionkey@0"
I20260925 02:47:13.080200 2495 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260925 02:47:13.082882 2510 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:47:13.084107 2495 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260925 02:47:13.084220 2495 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790304178325258-13942-0/minicluster-data/master-0/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790304178325258-13942-0/minicluster-data/master-0/wal
uuid: "648825c0b9ab4f78a8aeb52410287517"
format_stamp: "Formatted at 2026-09-25 02:47:13 on dist-test-slave-kznw"
server_key: "57f1a3132a5ba766508bb53eaa7db16e"
server_key_iv: "bb325b08e8d62cc1657b3769aa1a59ef"
server_key_version: "encryptionkey@0"
I20260925 02:47:13.084316 2495 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790304178325258-13942-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790304178325258-13942-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790304178325258-13942-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:47:13.088825 2495 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:47:13.091655 2495 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:47:13.091812 2495 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:47:13.098294 2495 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35117
I20260925 02:47:13.098337 2562 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35117 every 8 connection(s)
I20260925 02:47:13.099246 2495 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
I20260925 02:47:13.102015 2563 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:47:13.105475 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 2495
I20260925 02:47:13.105607 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790304178325258-13942-0/minicluster-data/master-0/wal/instance
I20260925 02:47:13.105847 13942 external_mini_cluster.cc:1428] Setting key 7ddb893900718d4c7aa19f1480579b44
I20260925 02:47:13.108048 2563 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 648825c0b9ab4f78a8aeb52410287517: Bootstrap starting.
I20260925 02:47:13.110402 2563 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 648825c0b9ab4f78a8aeb52410287517: Neither blocks nor log segments found. Creating new log.
I20260925 02:47:13.111066 2566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:13.107265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44279 (local address 127.0.0.1:35117)
0925 02:47:13.107745 (+ 480us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:13.107760 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:13.107796 (+ 36us) server_negotiation.cc:407] Connection header received
0925 02:47:13.108470 (+ 674us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:13.108490 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:13.108775 (+ 285us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:13.109092 (+ 317us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:47:13.109866 (+ 774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:47:13.109871 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:47:13.109882 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:47:13.109892 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:47:13.109944 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:47:13.109962 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:47:13.110055 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:13.110176 (+ 121us) server_negotiation.cc:299] Negotiation successful
0925 02:47:13.110329 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":327,"thread_start_us":149,"threads_started":1}
I20260925 02:47:13.111209 2563 log.cc:824] T 00000000000000000000000000000000 P 648825c0b9ab4f78a8aeb52410287517: Log is configured to *not* fsync() on all Append() calls
I20260925 02:47:13.113181 2563 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 648825c0b9ab4f78a8aeb52410287517: No bootstrap required, opened a new log
I20260925 02:47:13.113657 13942 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 02:47:13.117106 2566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:13.115566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37575 (local address 127.0.0.1:35117)
0925 02:47:13.115684 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:13.115688 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:13.115876 (+ 188us) server_negotiation.cc:407] Connection header received
0925 02:47:13.115982 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:13.115984 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:13.116015 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:13.116074 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:47:13.116766 (+ 692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:47:13.116770 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:47:13.116772 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:47:13.116783 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:47:13.116806 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:47:13.116816 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:47:13.116884 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:13.116955 (+ 71us) server_negotiation.cc:299] Negotiation successful
0925 02:47:13.116991 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260925 02:47:13.117211 2563 raft_consensus.cc:348] T 00000000000000000000000000000000 P 648825c0b9ab4f78a8aeb52410287517 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "648825c0b9ab4f78a8aeb52410287517" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35117 } }
I20260925 02:47:13.117470 2563 raft_consensus.cc:374] T 00000000000000000000000000000000 P 648825c0b9ab4f78a8aeb52410287517 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:47:13.117532 2563 raft_consensus.cc:729] T 00000000000000000000000000000000 P 648825c0b9ab4f78a8aeb52410287517 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 648825c0b9ab4f78a8aeb52410287517, State: Initialized, Role: FOLLOWER
I20260925 02:47:13.117990 2563 consensus_queue.cc:260] T 00000000000000000000000000000000 P 648825c0b9ab4f78a8aeb52410287517 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "648825c0b9ab4f78a8aeb52410287517" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35117 } }
I20260925 02:47:13.118158 2563 raft_consensus.cc:388] T 00000000000000000000000000000000 P 648825c0b9ab4f78a8aeb52410287517 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 02:47:13.118230 2563 raft_consensus.cc:482] T 00000000000000000000000000000000 P 648825c0b9ab4f78a8aeb52410287517 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 02:47:13.118330 2563 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 648825c0b9ab4f78a8aeb52410287517 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:47:13.119309 2563 raft_consensus.cc:504] T 00000000000000000000000000000000 P 648825c0b9ab4f78a8aeb52410287517 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "648825c0b9ab4f78a8aeb52410287517" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35117 } }
I20260925 02:47:13.119640 2563 leader_election.cc:302] T 00000000000000000000000000000000 P 648825c0b9ab4f78a8aeb52410287517 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 648825c0b9ab4f78a8aeb52410287517; no voters:
I20260925 02:47:13.119930 2563 leader_election.cc:288] T 00000000000000000000000000000000 P 648825c0b9ab4f78a8aeb52410287517 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 02:47:13.120062 2573 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 648825c0b9ab4f78a8aeb52410287517 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:47:13.120255 2573 raft_consensus.cc:686] T 00000000000000000000000000000000 P 648825c0b9ab4f78a8aeb52410287517 [term 1 LEADER]: Becoming Leader. State: Replica: 648825c0b9ab4f78a8aeb52410287517, State: Running, Role: LEADER
I20260925 02:47:13.120587 2573 consensus_queue.cc:237] T 00000000000000000000000000000000 P 648825c0b9ab4f78a8aeb52410287517 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "648825c0b9ab4f78a8aeb52410287517" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35117 } }
I20260925 02:47:13.121093 2563 sys_catalog.cc:564] T 00000000000000000000000000000000 P 648825c0b9ab4f78a8aeb52410287517 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:47:13.122138 2575 sys_catalog.cc:455] T 00000000000000000000000000000000 P 648825c0b9ab4f78a8aeb52410287517 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 648825c0b9ab4f78a8aeb52410287517. Latest consensus state: current_term: 1 leader_uuid: "648825c0b9ab4f78a8aeb52410287517" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "648825c0b9ab4f78a8aeb52410287517" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35117 } } }
I20260925 02:47:13.122284 2575 sys_catalog.cc:458] T 00000000000000000000000000000000 P 648825c0b9ab4f78a8aeb52410287517 [sys.catalog]: This master's current role is: LEADER
I20260925 02:47:13.122695 2582 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:47:13.122639 2574 sys_catalog.cc:455] T 00000000000000000000000000000000 P 648825c0b9ab4f78a8aeb52410287517 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "648825c0b9ab4f78a8aeb52410287517" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "648825c0b9ab4f78a8aeb52410287517" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35117 } } }
I20260925 02:47:13.123251 2574 sys_catalog.cc:458] T 00000000000000000000000000000000 P 648825c0b9ab4f78a8aeb52410287517 [sys.catalog]: This master's current role is: LEADER
I20260925 02:47:13.125864 2582 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:47:13.131147 2582 catalog_manager.cc:1347] Generated new cluster ID: 0d6589050ba147aba879b6e4dbdf3dfc
I20260925 02:47:13.131212 2582 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 02:47:13.147365 2582 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 02:47:13.147733 2582 catalog_manager.cc:1504] Loading token signing keys...
I20260925 02:47:13.160534 2582 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 648825c0b9ab4f78a8aeb52410287517: Generated new TSK 0
I20260925 02:47:13.161231 2582 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 02:47:13.168009 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 2495
2026-09-25T02:47:13Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1305 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790304178325258-13942-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 02:47:13 dist-test-slave-kznw krb5kdc[2594](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 02:47:13 dist-test-slave-kznw krb5kdc[2594](info): set up 2 sockets
Sep 25 02:47:13 dist-test-slave-kznw krb5kdc[2594](info): commencing operation
krb5kdc: starting...
W20260925 02:47:14.118453 13942 mini_kdc.cc:121] Time spent starting KDC: real 0.931s user 0.004s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790304178325258-13942-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790304178325258-13942-0/krb5kdc/test-user.keytab.
Sep 25 02:47:14 dist-test-slave-kznw krb5kdc[2594](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304434, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T02:47:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T02:47:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790304178325258-13942-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790304178325258-13942-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 02:47:14.232332 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790304178325258-13942-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790304178325258-13942-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790304178325258-13942-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37511
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790304178325258-13942-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44153
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37511
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:47:14.340801 2607 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:47:14.341078 2607 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:47:14.341131 2607 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:47:14.345147 2607 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:47:14.345201 2607 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:47:14.345219 2607 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:47:14.345234 2607 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:47:14.345247 2607 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:47:14.350514 2607 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44153
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790304178325258-13942-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790304178325258-13942-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37511
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790304178325258-13942-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37511
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2607
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790304178325258-13942-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:47:14.351542 2607 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:47:14.352495 2607 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:47:14.361145 2612 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:47:14.361212 2613 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:47:14.361156 2615 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:47:14.361578 2607 server_base.cc:1034] running on GCE node
I20260925 02:47:14.362124 2607 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:47:14.363277 2607 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:47:14.364444 2607 hybrid_clock.cc:648] HybridClock initialized: now 1790304434364423 us; error 31 us; skew 500 ppm
Sep 25 02:47:14 dist-test-slave-kznw krb5kdc[2594](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304434, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:47:14.367172 2607 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 02:47:14.368084 2607 webserver.cc:460] Webserver started at http://127.0.0.1:43849/ using document root <none> and password file <none>
I20260925 02:47:14.368613 2607 fs_manager.cc:359] Metadata directory not provided
I20260925 02:47:14.368677 2607 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:47:14.368886 2607 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:47:14.370879 2607 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790304178325258-13942-0/minicluster-data/master-0/data/instance:
uuid: "49de3de2acec49eaa65b588a6834ac01"
format_stamp: "Formatted at 2026-09-25 02:47:14 on dist-test-slave-kznw"
server_key: "55cc9dbb3afb3f8c4802694655c19d9e"
server_key_iv: "7cfb31d5ccde574593f0eaeadd8cbc53"
server_key_version: "encryptionkey@0"
I20260925 02:47:14.371474 2607 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790304178325258-13942-0/minicluster-data/master-0/wal/instance:
uuid: "49de3de2acec49eaa65b588a6834ac01"
format_stamp: "Formatted at 2026-09-25 02:47:14 on dist-test-slave-kznw"
server_key: "55cc9dbb3afb3f8c4802694655c19d9e"
server_key_iv: "7cfb31d5ccde574593f0eaeadd8cbc53"
server_key_version: "encryptionkey@0"
I20260925 02:47:14.375077 2607 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260925 02:47:14.377326 2622 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:47:14.378509 2607 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260925 02:47:14.378659 2607 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790304178325258-13942-0/minicluster-data/master-0/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790304178325258-13942-0/minicluster-data/master-0/wal
uuid: "49de3de2acec49eaa65b588a6834ac01"
format_stamp: "Formatted at 2026-09-25 02:47:14 on dist-test-slave-kznw"
server_key: "55cc9dbb3afb3f8c4802694655c19d9e"
server_key_iv: "7cfb31d5ccde574593f0eaeadd8cbc53"
server_key_version: "encryptionkey@0"
I20260925 02:47:14.378750 2607 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790304178325258-13942-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790304178325258-13942-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790304178325258-13942-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:47:14.392263 2607 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:47:14.395488 2607 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:47:14.395665 2607 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:47:14.402793 2607 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37511
I20260925 02:47:14.402897 2674 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37511 every 8 connection(s)
I20260925 02:47:14.403823 2607 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
I20260925 02:47:14.406726 2675 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:47:14.408190 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 2607
I20260925 02:47:14.408317 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790304178325258-13942-0/minicluster-data/master-0/wal/instance
I20260925 02:47:14.408641 13942 external_mini_cluster.cc:1428] Setting key 7fe6b79110d115a66228436c7febb7b4
I20260925 02:47:14.413365 2675 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49de3de2acec49eaa65b588a6834ac01: Bootstrap starting.
I20260925 02:47:14.415982 2675 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 49de3de2acec49eaa65b588a6834ac01: Neither blocks nor log segments found. Creating new log.
I20260925 02:47:14.416850 2675 log.cc:824] T 00000000000000000000000000000000 P 49de3de2acec49eaa65b588a6834ac01: Log is configured to *not* fsync() on all Append() calls
Sep 25 02:47:14 dist-test-slave-kznw krb5kdc[2594](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304434, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260925 02:47:14.419008 2675 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49de3de2acec49eaa65b588a6834ac01: No bootstrap required, opened a new log
I20260925 02:47:14.422762 2675 raft_consensus.cc:348] T 00000000000000000000000000000000 P 49de3de2acec49eaa65b588a6834ac01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49de3de2acec49eaa65b588a6834ac01" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37511 } }
I20260925 02:47:14.423059 2675 raft_consensus.cc:374] T 00000000000000000000000000000000 P 49de3de2acec49eaa65b588a6834ac01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:47:14.423132 2675 raft_consensus.cc:729] T 00000000000000000000000000000000 P 49de3de2acec49eaa65b588a6834ac01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49de3de2acec49eaa65b588a6834ac01, State: Initialized, Role: FOLLOWER
I20260925 02:47:14.423614 2675 consensus_queue.cc:260] T 00000000000000000000000000000000 P 49de3de2acec49eaa65b588a6834ac01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49de3de2acec49eaa65b588a6834ac01" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37511 } }
I20260925 02:47:14.423779 2675 raft_consensus.cc:388] T 00000000000000000000000000000000 P 49de3de2acec49eaa65b588a6834ac01 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 02:47:14.423894 2675 raft_consensus.cc:482] T 00000000000000000000000000000000 P 49de3de2acec49eaa65b588a6834ac01 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 02:47:14.424029 2675 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 49de3de2acec49eaa65b588a6834ac01 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:47:14.425143 2678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:14.410113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.34:55053 (local address 127.0.0.1:37511)
0925 02:47:14.410550 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:14.410570 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:14.410615 (+ 45us) server_negotiation.cc:407] Connection header received
0925 02:47:14.411519 (+ 904us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:14.411544 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:14.411995 (+ 451us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:14.412373 (+ 378us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:47:14.413094 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:14.414530 (+ 1436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:14.415512 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:14.416197 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:14.416363 (+ 166us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:14.419182 (+ 2819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:47:14.419188 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:47:14.419204 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:47:14.419235 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:47:14.421863 (+ 2628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:47:14.422342 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:47:14.422348 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:47:14.422354 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:47:14.422503 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:47:14.422877 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:47:14.422881 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:47:14.422882 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:47:14.423403 (+ 521us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:47:14.423625 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:14.423922 (+ 297us) server_negotiation.cc:299] Negotiation successful
0925 02:47:14.424156 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":285,"thread_start_us":114,"threads_started":1}
I20260925 02:47:14.425118 2675 raft_consensus.cc:504] T 00000000000000000000000000000000 P 49de3de2acec49eaa65b588a6834ac01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49de3de2acec49eaa65b588a6834ac01" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37511 } }
I20260925 02:47:14.425488 2675 leader_election.cc:302] T 00000000000000000000000000000000 P 49de3de2acec49eaa65b588a6834ac01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 49de3de2acec49eaa65b588a6834ac01; no voters:
I20260925 02:47:14.425776 2675 leader_election.cc:288] T 00000000000000000000000000000000 P 49de3de2acec49eaa65b588a6834ac01 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 02:47:14.425892 2680 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 49de3de2acec49eaa65b588a6834ac01 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:47:14.426049 2680 raft_consensus.cc:686] T 00000000000000000000000000000000 P 49de3de2acec49eaa65b588a6834ac01 [term 1 LEADER]: Becoming Leader. State: Replica: 49de3de2acec49eaa65b588a6834ac01, State: Running, Role: LEADER
I20260925 02:47:14.426401 2680 consensus_queue.cc:237] T 00000000000000000000000000000000 P 49de3de2acec49eaa65b588a6834ac01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49de3de2acec49eaa65b588a6834ac01" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37511 } }
I20260925 02:47:14.427103 2675 sys_catalog.cc:564] T 00000000000000000000000000000000 P 49de3de2acec49eaa65b588a6834ac01 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:47:14.427829 2682 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49de3de2acec49eaa65b588a6834ac01 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 49de3de2acec49eaa65b588a6834ac01. Latest consensus state: current_term: 1 leader_uuid: "49de3de2acec49eaa65b588a6834ac01" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49de3de2acec49eaa65b588a6834ac01" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37511 } } }
I20260925 02:47:14.427949 2682 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49de3de2acec49eaa65b588a6834ac01 [sys.catalog]: This master's current role is: LEADER
I20260925 02:47:14.428349 2689 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:47:14.428463 13942 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 02:47:14.431144 2681 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49de3de2acec49eaa65b588a6834ac01 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "49de3de2acec49eaa65b588a6834ac01" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49de3de2acec49eaa65b588a6834ac01" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37511 } } }
I20260925 02:47:14.431245 2681 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49de3de2acec49eaa65b588a6834ac01 [sys.catalog]: This master's current role is: LEADER
I20260925 02:47:14.431475 2689 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:47:14.440717 2689 catalog_manager.cc:1347] Generated new cluster ID: dca02dd6af7542a6a80f47bafb07649b
I20260925 02:47:14.440826 2689 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 02:47:14.443239 2678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:14.435391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.34:46715 (local address 127.0.0.1:37511)
0925 02:47:14.435497 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:14.435509 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:14.435842 (+ 333us) server_negotiation.cc:407] Connection header received
0925 02:47:14.435959 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:14.435962 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:14.436009 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:14.436072 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:47:14.436702 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:14.437213 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:14.437987 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:14.438231 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:14.438317 (+ 86us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:14.439861 (+ 1544us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:47:14.439864 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:47:14.439868 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:47:14.439893 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:47:14.441510 (+ 1617us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:47:14.442031 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:47:14.442035 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:47:14.442037 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:47:14.442124 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:47:14.442395 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:47:14.442398 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:47:14.442400 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:47:14.442608 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:47:14.442753 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:14.442987 (+ 234us) server_negotiation.cc:299] Negotiation successful
0925 02:47:14.443084 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260925 02:47:14.461114 2689 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 02:47:14.462366 2689 catalog_manager.cc:1504] Loading token signing keys...
I20260925 02:47:14.475564 2689 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 49de3de2acec49eaa65b588a6834ac01: Generated new TSK 0
I20260925 02:47:14.476406 2689 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 02:47:14.488304 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 2607
W20260925 02:47:14.493559 2592 connection.cc:535] client connection to 127.0.0.1:37511 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37511): Connection reset by peer (error 104)
2026-09-25T02:47:14Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1320 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790304178325258-13942-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 02:47:14 dist-test-slave-kznw krb5kdc[2706](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 02:47:14 dist-test-slave-kznw krb5kdc[2706](info): set up 2 sockets
Sep 25 02:47:14 dist-test-slave-kznw krb5kdc[2706](info): commencing operation
krb5kdc: starting...
W20260925 02:47:15.450448 13942 mini_kdc.cc:121] Time spent starting KDC: real 0.943s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790304178325258-13942-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790304178325258-13942-0/krb5kdc/test-user.keytab.
Sep 25 02:47:15 dist-test-slave-kznw krb5kdc[2706](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304435, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T02:47:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T02:47:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790304178325258-13942-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790304178325258-13942-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 02:47:15.570334 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790304178325258-13942-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790304178325258-13942-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790304178325258-13942-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33223
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790304178325258-13942-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33223
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:47:15.679982 2719 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:47:15.680266 2719 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:47:15.680323 2719 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:47:15.684266 2719 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:47:15.684327 2719 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:47:15.684352 2719 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:47:15.684376 2719 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:47:15.684403 2719 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:47:15.689643 2719 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790304178325258-13942-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790304178325258-13942-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33223
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790304178325258-13942-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33223
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2719
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790304178325258-13942-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:47:15.690712 2719 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:47:15.691800 2719 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:47:15.699982 2725 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:47:15.700006 2724 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:47:15.699994 2727 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:47:15.700351 2719 server_base.cc:1034] running on GCE node
I20260925 02:47:15.700896 2719 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:47:15.701958 2719 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:47:15.703114 2719 hybrid_clock.cc:648] HybridClock initialized: now 1790304435703096 us; error 33 us; skew 500 ppm
Sep 25 02:47:15 dist-test-slave-kznw krb5kdc[2706](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304435, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:47:15.705868 2719 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 02:47:15.706820 2719 webserver.cc:460] Webserver started at http://127.0.0.1:35243/ using document root <none> and password file <none>
I20260925 02:47:15.707386 2719 fs_manager.cc:359] Metadata directory not provided
I20260925 02:47:15.707464 2719 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:47:15.707664 2719 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:47:15.709703 2719 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790304178325258-13942-0/minicluster-data/master-0/data/instance:
uuid: "91c0ce321c0a4d21bf6cfc506d75f309"
format_stamp: "Formatted at 2026-09-25 02:47:15 on dist-test-slave-kznw"
server_key: "5a4f653db8585d2330bcacb114b877e9"
server_key_iv: "f64d66fa2bb7fbda8475b192be1a78c1"
server_key_version: "encryptionkey@0"
I20260925 02:47:15.710258 2719 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790304178325258-13942-0/minicluster-data/master-0/wal/instance:
uuid: "91c0ce321c0a4d21bf6cfc506d75f309"
format_stamp: "Formatted at 2026-09-25 02:47:15 on dist-test-slave-kznw"
server_key: "5a4f653db8585d2330bcacb114b877e9"
server_key_iv: "f64d66fa2bb7fbda8475b192be1a78c1"
server_key_version: "encryptionkey@0"
I20260925 02:47:15.713985 2719 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260925 02:47:15.716334 2734 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:47:15.717549 2719 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260925 02:47:15.717662 2719 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790304178325258-13942-0/minicluster-data/master-0/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790304178325258-13942-0/minicluster-data/master-0/wal
uuid: "91c0ce321c0a4d21bf6cfc506d75f309"
format_stamp: "Formatted at 2026-09-25 02:47:15 on dist-test-slave-kznw"
server_key: "5a4f653db8585d2330bcacb114b877e9"
server_key_iv: "f64d66fa2bb7fbda8475b192be1a78c1"
server_key_version: "encryptionkey@0"
I20260925 02:47:15.717751 2719 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790304178325258-13942-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790304178325258-13942-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790304178325258-13942-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:47:15.730082 2719 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:47:15.733075 2719 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:47:15.733260 2719 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:47:15.740192 2719 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33223
I20260925 02:47:15.740208 2786 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33223 every 8 connection(s)
I20260925 02:47:15.741163 2719 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
I20260925 02:47:15.743973 2787 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:47:15.746454 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 2719
I20260925 02:47:15.746587 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790304178325258-13942-0/minicluster-data/master-0/wal/instance
I20260925 02:47:15.746846 13942 external_mini_cluster.cc:1428] Setting key 70654f17927277091a96869b3e925dc3
I20260925 02:47:15.750025 2787 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91c0ce321c0a4d21bf6cfc506d75f309: Bootstrap starting.
I20260925 02:47:15.752712 2787 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 91c0ce321c0a4d21bf6cfc506d75f309: Neither blocks nor log segments found. Creating new log.
I20260925 02:47:15.753456 2787 log.cc:824] T 00000000000000000000000000000000 P 91c0ce321c0a4d21bf6cfc506d75f309: Log is configured to *not* fsync() on all Append() calls
I20260925 02:47:15.755594 2787 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91c0ce321c0a4d21bf6cfc506d75f309: No bootstrap required, opened a new log
I20260925 02:47:15.757046 2790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:15.748385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.34:59161 (local address 127.0.0.1:33223)
0925 02:47:15.748830 (+ 445us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:15.748850 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:15.748894 (+ 44us) server_negotiation.cc:407] Connection header received
0925 02:47:15.749692 (+ 798us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:15.749716 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:15.750153 (+ 437us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:15.750526 (+ 373us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:47:15.750962 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:15.752423 (+ 1461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:15.753401 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:15.754034 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:15.754156 (+ 122us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:15.755028 (+ 872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:47:15.755034 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:47:15.755050 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:47:15.755549 (+ 499us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:47:15.755624 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:47:15.755658 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:47:15.755792 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:15.755975 (+ 183us) server_negotiation.cc:299] Negotiation successful
0925 02:47:15.756116 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":287,"thread_start_us":123,"threads_started":1}
I20260925 02:47:15.759480 2787 raft_consensus.cc:348] T 00000000000000000000000000000000 P 91c0ce321c0a4d21bf6cfc506d75f309 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91c0ce321c0a4d21bf6cfc506d75f309" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33223 } }
I20260925 02:47:15.759693 2787 raft_consensus.cc:374] T 00000000000000000000000000000000 P 91c0ce321c0a4d21bf6cfc506d75f309 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:47:15.759730 2787 raft_consensus.cc:729] T 00000000000000000000000000000000 P 91c0ce321c0a4d21bf6cfc506d75f309 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91c0ce321c0a4d21bf6cfc506d75f309, State: Initialized, Role: FOLLOWER
I20260925 02:47:15.759963 13942 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 02:47:15.760197 2787 consensus_queue.cc:260] T 00000000000000000000000000000000 P 91c0ce321c0a4d21bf6cfc506d75f309 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91c0ce321c0a4d21bf6cfc506d75f309" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33223 } }
I20260925 02:47:15.760349 2787 raft_consensus.cc:388] T 00000000000000000000000000000000 P 91c0ce321c0a4d21bf6cfc506d75f309 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 02:47:15.760407 2787 raft_consensus.cc:482] T 00000000000000000000000000000000 P 91c0ce321c0a4d21bf6cfc506d75f309 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 02:47:15.760507 2787 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 91c0ce321c0a4d21bf6cfc506d75f309 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:47:15.761656 2787 raft_consensus.cc:504] T 00000000000000000000000000000000 P 91c0ce321c0a4d21bf6cfc506d75f309 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91c0ce321c0a4d21bf6cfc506d75f309" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33223 } }
I20260925 02:47:15.762032 2787 leader_election.cc:302] T 00000000000000000000000000000000 P 91c0ce321c0a4d21bf6cfc506d75f309 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 91c0ce321c0a4d21bf6cfc506d75f309; no voters:
I20260925 02:47:15.762367 2787 leader_election.cc:288] T 00000000000000000000000000000000 P 91c0ce321c0a4d21bf6cfc506d75f309 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 02:47:15.762467 2796 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 91c0ce321c0a4d21bf6cfc506d75f309 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:47:15.762758 2796 raft_consensus.cc:686] T 00000000000000000000000000000000 P 91c0ce321c0a4d21bf6cfc506d75f309 [term 1 LEADER]: Becoming Leader. State: Replica: 91c0ce321c0a4d21bf6cfc506d75f309, State: Running, Role: LEADER
I20260925 02:47:15.763176 2796 consensus_queue.cc:237] T 00000000000000000000000000000000 P 91c0ce321c0a4d21bf6cfc506d75f309 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91c0ce321c0a4d21bf6cfc506d75f309" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33223 } }
I20260925 02:47:15.763785 2787 sys_catalog.cc:564] T 00000000000000000000000000000000 P 91c0ce321c0a4d21bf6cfc506d75f309 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:47:15.764932 2799 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91c0ce321c0a4d21bf6cfc506d75f309 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 91c0ce321c0a4d21bf6cfc506d75f309. Latest consensus state: current_term: 1 leader_uuid: "91c0ce321c0a4d21bf6cfc506d75f309" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91c0ce321c0a4d21bf6cfc506d75f309" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33223 } } }
I20260925 02:47:15.765074 2799 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91c0ce321c0a4d21bf6cfc506d75f309 [sys.catalog]: This master's current role is: LEADER
I20260925 02:47:15.765401 2798 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91c0ce321c0a4d21bf6cfc506d75f309 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "91c0ce321c0a4d21bf6cfc506d75f309" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91c0ce321c0a4d21bf6cfc506d75f309" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33223 } } }
I20260925 02:47:15.765501 2798 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91c0ce321c0a4d21bf6cfc506d75f309 [sys.catalog]: This master's current role is: LEADER
I20260925 02:47:15.765511 2806 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:47:15.767838 2790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:15.762513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.34:48813 (local address 127.0.0.1:33223)
0925 02:47:15.762625 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:15.762629 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:15.762785 (+ 156us) server_negotiation.cc:407] Connection header received
0925 02:47:15.763041 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:15.763044 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:15.763097 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:15.763178 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:47:15.763600 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:15.764390 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:15.765409 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:15.765732 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:15.765826 (+ 94us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:15.766809 (+ 983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:47:15.766812 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:47:15.766815 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:47:15.766828 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:47:15.766856 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:47:15.766868 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:47:15.767018 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:15.767675 (+ 657us) server_negotiation.cc:299] Negotiation successful
0925 02:47:15.767713 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260925 02:47:15.769333 2806 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:47:15.774808 2806 catalog_manager.cc:1347] Generated new cluster ID: 705f0eea371c4f509a3584468c17bd18
I20260925 02:47:15.774878 2806 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 02:47:15.789487 2806 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 02:47:15.790756 2806 catalog_manager.cc:1504] Loading token signing keys...
I20260925 02:47:15.800403 2806 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 91c0ce321c0a4d21bf6cfc506d75f309: Generated new TSK 0
I20260925 02:47:15.801196 2806 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 02:47:15.806039 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 2719
W20260925 02:47:15.811388 2704 connection.cc:535] client connection to 127.0.0.1:33223 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33223): Connection reset by peer (error 104)
2026-09-25T02:47:15Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1317 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790304178325258-13942-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 02:47:15 dist-test-slave-kznw krb5kdc[2818](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 02:47:15 dist-test-slave-kznw krb5kdc[2818](info): set up 2 sockets
Sep 25 02:47:15 dist-test-slave-kznw krb5kdc[2818](info): commencing operation
krb5kdc: starting...
W20260925 02:47:16.762568 13942 mini_kdc.cc:121] Time spent starting KDC: real 0.937s user 0.005s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790304178325258-13942-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790304178325258-13942-0/krb5kdc/test-user.keytab.
Sep 25 02:47:16 dist-test-slave-kznw krb5kdc[2818](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304436, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T02:47:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T02:47:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790304178325258-13942-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790304178325258-13942-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 02:47:16.881156 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790304178325258-13942-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790304178325258-13942-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790304178325258-13942-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42925
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790304178325258-13942-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36507
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42925
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:47:16.990243 2831 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:47:16.990549 2831 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:47:16.990609 2831 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:47:16.994556 2831 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:47:16.994616 2831 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:47:16.994642 2831 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:47:16.994663 2831 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:47:16.994689 2831 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:47:16.999960 2831 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36507
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790304178325258-13942-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790304178325258-13942-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42925
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790304178325258-13942-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42925
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2831
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790304178325258-13942-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:47:17.001013 2831 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:47:17.002027 2831 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:47:17.010298 2839 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:47:17.010300 2836 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:47:17.010375 2831 server_base.cc:1034] running on GCE node
W20260925 02:47:17.010301 2837 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:47:17.011178 2831 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:47:17.012293 2831 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:47:17.013447 2831 hybrid_clock.cc:648] HybridClock initialized: now 1790304437013424 us; error 39 us; skew 500 ppm
Sep 25 02:47:17 dist-test-slave-kznw krb5kdc[2818](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304437, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:47:17.019496 2831 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 02:47:17.020430 2831 webserver.cc:460] Webserver started at http://127.0.0.1:33261/ using document root <none> and password file <none>
I20260925 02:47:17.020977 2831 fs_manager.cc:359] Metadata directory not provided
I20260925 02:47:17.021042 2831 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:47:17.021245 2831 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:47:17.023350 2831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790304178325258-13942-0/minicluster-data/master-0/data/instance:
uuid: "f89fe0ccf82441a09dd9bc108bbaed11"
format_stamp: "Formatted at 2026-09-25 02:47:17 on dist-test-slave-kznw"
server_key: "3d84491f8b20797b890737f1017dcabf"
server_key_iv: "918ed78571ce4dc7e88e4e2dc55f5634"
server_key_version: "encryptionkey@0"
I20260925 02:47:17.023923 2831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790304178325258-13942-0/minicluster-data/master-0/wal/instance:
uuid: "f89fe0ccf82441a09dd9bc108bbaed11"
format_stamp: "Formatted at 2026-09-25 02:47:17 on dist-test-slave-kznw"
server_key: "3d84491f8b20797b890737f1017dcabf"
server_key_iv: "918ed78571ce4dc7e88e4e2dc55f5634"
server_key_version: "encryptionkey@0"
I20260925 02:47:17.027877 2831 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260925 02:47:17.030364 2846 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:47:17.031646 2831 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260925 02:47:17.031764 2831 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790304178325258-13942-0/minicluster-data/master-0/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790304178325258-13942-0/minicluster-data/master-0/wal
uuid: "f89fe0ccf82441a09dd9bc108bbaed11"
format_stamp: "Formatted at 2026-09-25 02:47:17 on dist-test-slave-kznw"
server_key: "3d84491f8b20797b890737f1017dcabf"
server_key_iv: "918ed78571ce4dc7e88e4e2dc55f5634"
server_key_version: "encryptionkey@0"
I20260925 02:47:17.031862 2831 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790304178325258-13942-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790304178325258-13942-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790304178325258-13942-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:47:17.036352 2831 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:47:17.039240 2831 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:47:17.039408 2831 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:47:17.046010 2831 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42925
I20260925 02:47:17.046053 2898 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42925 every 8 connection(s)
I20260925 02:47:17.046960 2831 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
I20260925 02:47:17.047252 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 2831
I20260925 02:47:17.047371 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790304178325258-13942-0/minicluster-data/master-0/wal/instance
I20260925 02:47:17.047688 13942 external_mini_cluster.cc:1428] Setting key 17ae6335a10a5351a32d1ddb2b57e095
I20260925 02:47:17.049950 2899 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:47:17.052685 2902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:17.049038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.34:60807 (local address 127.0.0.1:42925)
0925 02:47:17.049466 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:17.049482 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:17.049517 (+ 35us) server_negotiation.cc:407] Connection header received
0925 02:47:17.050192 (+ 675us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:17.050210 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:17.050496 (+ 286us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:17.050808 (+ 312us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:47:17.051572 (+ 764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:47:17.051577 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:47:17.051588 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:47:17.051598 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:47:17.051652 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:47:17.051670 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:47:17.051793 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:17.051860 (+ 67us) server_negotiation.cc:299] Negotiation successful
0925 02:47:17.052025 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":286,"thread_start_us":125,"threads_started":1}
I20260925 02:47:17.055636 13942 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 02:47:17.056022 2899 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f89fe0ccf82441a09dd9bc108bbaed11: Bootstrap starting.
I20260925 02:47:17.058346 2899 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f89fe0ccf82441a09dd9bc108bbaed11: Neither blocks nor log segments found. Creating new log.
I20260925 02:47:17.059237 2902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:17.057586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.34:50657 (local address 127.0.0.1:42925)
0925 02:47:17.057701 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:17.057705 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:17.057973 (+ 268us) server_negotiation.cc:407] Connection header received
0925 02:47:17.058062 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:17.058064 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:17.058095 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:17.058162 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:47:17.058836 (+ 674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:47:17.058839 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:47:17.058841 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:47:17.058851 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:47:17.058876 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:47:17.058898 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:47:17.059000 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:17.059088 (+ 88us) server_negotiation.cc:299] Negotiation successful
0925 02:47:17.059129 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260925 02:47:17.059263 2899 log.cc:824] T 00000000000000000000000000000000 P f89fe0ccf82441a09dd9bc108bbaed11: Log is configured to *not* fsync() on all Append() calls
I20260925 02:47:17.061168 2899 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f89fe0ccf82441a09dd9bc108bbaed11: No bootstrap required, opened a new log
I20260925 02:47:17.064621 2899 raft_consensus.cc:348] T 00000000000000000000000000000000 P f89fe0ccf82441a09dd9bc108bbaed11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f89fe0ccf82441a09dd9bc108bbaed11" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42925 } }
I20260925 02:47:17.064844 2899 raft_consensus.cc:374] T 00000000000000000000000000000000 P f89fe0ccf82441a09dd9bc108bbaed11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:47:17.064913 2899 raft_consensus.cc:729] T 00000000000000000000000000000000 P f89fe0ccf82441a09dd9bc108bbaed11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f89fe0ccf82441a09dd9bc108bbaed11, State: Initialized, Role: FOLLOWER
I20260925 02:47:17.065315 2899 consensus_queue.cc:260] T 00000000000000000000000000000000 P f89fe0ccf82441a09dd9bc108bbaed11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f89fe0ccf82441a09dd9bc108bbaed11" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42925 } }
I20260925 02:47:17.065445 2899 raft_consensus.cc:388] T 00000000000000000000000000000000 P f89fe0ccf82441a09dd9bc108bbaed11 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 02:47:17.065510 2899 raft_consensus.cc:482] T 00000000000000000000000000000000 P f89fe0ccf82441a09dd9bc108bbaed11 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 02:47:17.065608 2899 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f89fe0ccf82441a09dd9bc108bbaed11 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:47:17.066560 2899 raft_consensus.cc:504] T 00000000000000000000000000000000 P f89fe0ccf82441a09dd9bc108bbaed11 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f89fe0ccf82441a09dd9bc108bbaed11" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42925 } }
I20260925 02:47:17.066883 2899 leader_election.cc:302] T 00000000000000000000000000000000 P f89fe0ccf82441a09dd9bc108bbaed11 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f89fe0ccf82441a09dd9bc108bbaed11; no voters:
I20260925 02:47:17.067236 2899 leader_election.cc:288] T 00000000000000000000000000000000 P f89fe0ccf82441a09dd9bc108bbaed11 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 02:47:17.067425 2909 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f89fe0ccf82441a09dd9bc108bbaed11 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:47:17.067695 2909 raft_consensus.cc:686] T 00000000000000000000000000000000 P f89fe0ccf82441a09dd9bc108bbaed11 [term 1 LEADER]: Becoming Leader. State: Replica: f89fe0ccf82441a09dd9bc108bbaed11, State: Running, Role: LEADER
I20260925 02:47:17.068003 2909 consensus_queue.cc:237] T 00000000000000000000000000000000 P f89fe0ccf82441a09dd9bc108bbaed11 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f89fe0ccf82441a09dd9bc108bbaed11" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42925 } }
I20260925 02:47:17.068400 2899 sys_catalog.cc:564] T 00000000000000000000000000000000 P f89fe0ccf82441a09dd9bc108bbaed11 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:47:17.069515 2911 sys_catalog.cc:455] T 00000000000000000000000000000000 P f89fe0ccf82441a09dd9bc108bbaed11 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f89fe0ccf82441a09dd9bc108bbaed11. Latest consensus state: current_term: 1 leader_uuid: "f89fe0ccf82441a09dd9bc108bbaed11" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f89fe0ccf82441a09dd9bc108bbaed11" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42925 } } }
I20260925 02:47:17.069649 2911 sys_catalog.cc:458] T 00000000000000000000000000000000 P f89fe0ccf82441a09dd9bc108bbaed11 [sys.catalog]: This master's current role is: LEADER
I20260925 02:47:17.069993 2910 sys_catalog.cc:455] T 00000000000000000000000000000000 P f89fe0ccf82441a09dd9bc108bbaed11 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f89fe0ccf82441a09dd9bc108bbaed11" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f89fe0ccf82441a09dd9bc108bbaed11" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42925 } } }
I20260925 02:47:17.070084 2910 sys_catalog.cc:458] T 00000000000000000000000000000000 P f89fe0ccf82441a09dd9bc108bbaed11 [sys.catalog]: This master's current role is: LEADER
I20260925 02:47:17.070441 2920 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:47:17.073589 2920 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:47:17.079530 2920 catalog_manager.cc:1347] Generated new cluster ID: ceac0011afb64b0d9394a30725a19f0d
I20260925 02:47:17.079607 2920 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 02:47:17.101584 2920 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 02:47:17.102007 2920 catalog_manager.cc:1504] Loading token signing keys...
I20260925 02:47:17.108632 2920 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f89fe0ccf82441a09dd9bc108bbaed11: Generated new TSK 0
I20260925 02:47:17.109462 2920 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 02:47:17.116806 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 2831
2026-09-25T02:47:17Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1311 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790304178325258-13942-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 02:47:17 dist-test-slave-kznw krb5kdc[2930](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 02:47:17 dist-test-slave-kznw krb5kdc[2930](info): set up 2 sockets
Sep 25 02:47:17 dist-test-slave-kznw krb5kdc[2930](info): commencing operation
krb5kdc: starting...
W20260925 02:47:18.078512 13942 mini_kdc.cc:121] Time spent starting KDC: real 0.942s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790304178325258-13942-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790304178325258-13942-0/krb5kdc/test-user.keytab.
Sep 25 02:47:18 dist-test-slave-kznw krb5kdc[2930](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304438, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T02:47:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T02:47:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790304178325258-13942-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790304178325258-13942-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 02:47:18.193995 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790304178325258-13942-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790304178325258-13942-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790304178325258-13942-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34257
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790304178325258-13942-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34257
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:47:18.300385 2943 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:47:18.300745 2943 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:47:18.300865 2943 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:47:18.306499 2943 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:47:18.306567 2943 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:47:18.306592 2943 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:47:18.306624 2943 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:47:18.306653 2943 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:47:18.313203 2943 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790304178325258-13942-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790304178325258-13942-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34257
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790304178325258-13942-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34257
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2943
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790304178325258-13942-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:47:18.314679 2943 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:47:18.316085 2943 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:47:18.324615 2951 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:47:18.324681 2949 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:47:18.324641 2943 server_base.cc:1034] running on GCE node
W20260925 02:47:18.324754 2948 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:47:18.325429 2943 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:47:18.326516 2943 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:47:18.327677 2943 hybrid_clock.cc:648] HybridClock initialized: now 1790304438327650 us; error 35 us; skew 500 ppm
Sep 25 02:47:18 dist-test-slave-kznw krb5kdc[2930](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304438, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:47:18.330338 2943 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 02:47:18.331281 2943 webserver.cc:460] Webserver started at http://127.0.0.1:38607/ using document root <none> and password file <none>
I20260925 02:47:18.331779 2943 fs_manager.cc:359] Metadata directory not provided
I20260925 02:47:18.331837 2943 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:47:18.332016 2943 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:47:18.333984 2943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790304178325258-13942-0/minicluster-data/master-0/data/instance:
uuid: "6d39e9f8c106427e9045ed0fa0606fb6"
format_stamp: "Formatted at 2026-09-25 02:47:18 on dist-test-slave-kznw"
server_key: "a072ee184ea3130c7bd7f3a96a61ea94"
server_key_iv: "ad593ca81fbc53b15afb121b5990f7ed"
server_key_version: "encryptionkey@0"
I20260925 02:47:18.334482 2943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790304178325258-13942-0/minicluster-data/master-0/wal/instance:
uuid: "6d39e9f8c106427e9045ed0fa0606fb6"
format_stamp: "Formatted at 2026-09-25 02:47:18 on dist-test-slave-kznw"
server_key: "a072ee184ea3130c7bd7f3a96a61ea94"
server_key_iv: "ad593ca81fbc53b15afb121b5990f7ed"
server_key_version: "encryptionkey@0"
I20260925 02:47:18.338027 2943 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260925 02:47:18.340348 2958 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:47:18.341836 2943 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260925 02:47:18.341965 2943 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790304178325258-13942-0/minicluster-data/master-0/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790304178325258-13942-0/minicluster-data/master-0/wal
uuid: "6d39e9f8c106427e9045ed0fa0606fb6"
format_stamp: "Formatted at 2026-09-25 02:47:18 on dist-test-slave-kznw"
server_key: "a072ee184ea3130c7bd7f3a96a61ea94"
server_key_iv: "ad593ca81fbc53b15afb121b5990f7ed"
server_key_version: "encryptionkey@0"
I20260925 02:47:18.342053 2943 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790304178325258-13942-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790304178325258-13942-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790304178325258-13942-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:47:18.355350 2943 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:47:18.358124 2943 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:47:18.358273 2943 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:47:18.364833 2943 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34257
I20260925 02:47:18.364881 3010 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34257 every 8 connection(s)
I20260925 02:47:18.365751 2943 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
I20260925 02:47:18.368559 3011 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:47:18.369506 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 2943
I20260925 02:47:18.369649 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790304178325258-13942-0/minicluster-data/master-0/wal/instance
I20260925 02:47:18.369953 13942 external_mini_cluster.cc:1428] Setting key 8a58c4326489392651fdd983404bc0be
I20260925 02:47:18.374708 3011 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d39e9f8c106427e9045ed0fa0606fb6: Bootstrap starting.
I20260925 02:47:18.377285 3011 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6d39e9f8c106427e9045ed0fa0606fb6: Neither blocks nor log segments found. Creating new log.
I20260925 02:47:18.378166 3011 log.cc:824] T 00000000000000000000000000000000 P 6d39e9f8c106427e9045ed0fa0606fb6: Log is configured to *not* fsync() on all Append() calls
Sep 25 02:47:18 dist-test-slave-kznw krb5kdc[2930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304438, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260925 02:47:18.380133 3011 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d39e9f8c106427e9045ed0fa0606fb6: No bootstrap required, opened a new log
I20260925 02:47:18.383535 3011 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6d39e9f8c106427e9045ed0fa0606fb6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d39e9f8c106427e9045ed0fa0606fb6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34257 } }
I20260925 02:47:18.383751 3011 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6d39e9f8c106427e9045ed0fa0606fb6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:47:18.383806 3011 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6d39e9f8c106427e9045ed0fa0606fb6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d39e9f8c106427e9045ed0fa0606fb6, State: Initialized, Role: FOLLOWER
I20260925 02:47:18.383917 3014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:18.371306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.34:59235 (local address 127.0.0.1:34257)
0925 02:47:18.371708 (+ 402us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:18.371723 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:18.371757 (+ 34us) server_negotiation.cc:407] Connection header received
0925 02:47:18.372390 (+ 633us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:18.372405 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:18.372746 (+ 341us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:18.373021 (+ 275us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:47:18.373880 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:18.374958 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:18.376067 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:18.376538 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:18.376650 (+ 112us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:18.379412 (+ 2762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:47:18.379417 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:47:18.379432 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:47:18.379457 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:47:18.381052 (+ 1595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:47:18.381534 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:47:18.381538 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:47:18.381542 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:47:18.381649 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:47:18.382034 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:47:18.382037 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:47:18.382038 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:47:18.382412 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:47:18.382622 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:18.382962 (+ 340us) server_negotiation.cc:299] Negotiation successful
0925 02:47:18.383164 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":272,"thread_start_us":117,"threads_started":1}
I20260925 02:47:18.384279 3011 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6d39e9f8c106427e9045ed0fa0606fb6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d39e9f8c106427e9045ed0fa0606fb6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34257 } }
I20260925 02:47:18.384419 3011 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6d39e9f8c106427e9045ed0fa0606fb6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 02:47:18.384496 3011 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6d39e9f8c106427e9045ed0fa0606fb6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 02:47:18.384593 3011 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6d39e9f8c106427e9045ed0fa0606fb6 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:47:18.385382 3011 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6d39e9f8c106427e9045ed0fa0606fb6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d39e9f8c106427e9045ed0fa0606fb6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34257 } }
I20260925 02:47:18.385701 3011 leader_election.cc:302] T 00000000000000000000000000000000 P 6d39e9f8c106427e9045ed0fa0606fb6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6d39e9f8c106427e9045ed0fa0606fb6; no voters:
I20260925 02:47:18.386047 3011 leader_election.cc:288] T 00000000000000000000000000000000 P 6d39e9f8c106427e9045ed0fa0606fb6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 02:47:18.386126 3016 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6d39e9f8c106427e9045ed0fa0606fb6 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:47:18.386324 3016 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6d39e9f8c106427e9045ed0fa0606fb6 [term 1 LEADER]: Becoming Leader. State: Replica: 6d39e9f8c106427e9045ed0fa0606fb6, State: Running, Role: LEADER
I20260925 02:47:18.386690 3016 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6d39e9f8c106427e9045ed0fa0606fb6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d39e9f8c106427e9045ed0fa0606fb6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34257 } }
I20260925 02:47:18.386966 13942 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 02:47:18.387288 3011 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6d39e9f8c106427e9045ed0fa0606fb6 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:47:18.388427 3017 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d39e9f8c106427e9045ed0fa0606fb6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6d39e9f8c106427e9045ed0fa0606fb6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d39e9f8c106427e9045ed0fa0606fb6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34257 } } }
I20260925 02:47:18.388550 3017 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d39e9f8c106427e9045ed0fa0606fb6 [sys.catalog]: This master's current role is: LEADER
I20260925 02:47:18.388881 3018 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d39e9f8c106427e9045ed0fa0606fb6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6d39e9f8c106427e9045ed0fa0606fb6. Latest consensus state: current_term: 1 leader_uuid: "6d39e9f8c106427e9045ed0fa0606fb6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d39e9f8c106427e9045ed0fa0606fb6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34257 } } }
I20260925 02:47:18.389055 3018 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d39e9f8c106427e9045ed0fa0606fb6 [sys.catalog]: This master's current role is: LEADER
I20260925 02:47:18.389827 3030 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:47:18.392728 3030 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:47:18.396984 3014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:18.389511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.34:56647 (local address 127.0.0.1:34257)
0925 02:47:18.389616 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:18.389621 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:18.389875 (+ 254us) server_negotiation.cc:407] Connection header received
0925 02:47:18.390005 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:18.390008 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:18.390057 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:18.390120 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:47:18.390736 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:18.391310 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:18.392099 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:18.392313 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:18.392405 (+ 92us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:18.393619 (+ 1214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:47:18.393622 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:47:18.393624 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:47:18.393646 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:47:18.395041 (+ 1395us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:47:18.395586 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:47:18.395590 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:47:18.395592 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:47:18.395677 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:47:18.396005 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:47:18.396009 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:47:18.396011 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:47:18.396213 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:47:18.396346 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:18.396724 (+ 378us) server_negotiation.cc:299] Negotiation successful
0925 02:47:18.396822 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260925 02:47:18.398594 3030 catalog_manager.cc:1347] Generated new cluster ID: 2d3861d210fa409f8b5d4fc54c2b4d3d
I20260925 02:47:18.398655 3030 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 02:47:18.414161 3030 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 02:47:18.415308 3030 catalog_manager.cc:1504] Loading token signing keys...
I20260925 02:47:18.430735 3030 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6d39e9f8c106427e9045ed0fa0606fb6: Generated new TSK 0
I20260925 02:47:18.431427 3030 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 02:47:18.435043 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 2943
W20260925 02:47:18.440557 2928 connection.cc:535] client connection to 127.0.0.1:34257 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34257): Connection reset by peer (error 104)
2026-09-25T02:47:18Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1317 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790304178325258-13942-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 02:47:18 dist-test-slave-kznw krb5kdc[3042](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 02:47:18 dist-test-slave-kznw krb5kdc[3042](info): set up 2 sockets
Sep 25 02:47:18 dist-test-slave-kznw krb5kdc[3042](info): commencing operation
krb5kdc: starting...
W20260925 02:47:19.390610 13942 mini_kdc.cc:121] Time spent starting KDC: real 0.936s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790304178325258-13942-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790304178325258-13942-0/krb5kdc/test-user.keytab.
Sep 25 02:47:19 dist-test-slave-kznw krb5kdc[3042](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304439, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T02:47:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T02:47:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790304178325258-13942-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790304178325258-13942-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 02:47:19.511842 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790304178325258-13942-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790304178325258-13942-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790304178325258-13942-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45011
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790304178325258-13942-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45011
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:47:19.620800 3055 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:47:19.621061 3055 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:47:19.621102 3055 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:47:19.626116 3055 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:47:19.626199 3055 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:47:19.626222 3055 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:47:19.626243 3055 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:47:19.626271 3055 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:47:19.631598 3055 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790304178325258-13942-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790304178325258-13942-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45011
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790304178325258-13942-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45011
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3055
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790304178325258-13942-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:47:19.632587 3055 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:47:19.633559 3055 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:47:19.641927 3063 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:47:19.641987 3055 server_base.cc:1034] running on GCE node
W20260925 02:47:19.641981 3061 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:47:19.641947 3060 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:47:19.642751 3055 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:47:19.643903 3055 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:47:19.645068 3055 hybrid_clock.cc:648] HybridClock initialized: now 1790304439645045 us; error 35 us; skew 500 ppm
Sep 25 02:47:19 dist-test-slave-kznw krb5kdc[3042](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304439, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:47:19.647663 3055 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 02:47:19.648535 3055 webserver.cc:460] Webserver started at http://127.0.0.1:45347/ using document root <none> and password file <none>
I20260925 02:47:19.649097 3055 fs_manager.cc:359] Metadata directory not provided
I20260925 02:47:19.649168 3055 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:47:19.649371 3055 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:47:19.651440 3055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790304178325258-13942-0/minicluster-data/master-0/data/instance:
uuid: "03992beb9c554e7886160d5940498e4c"
format_stamp: "Formatted at 2026-09-25 02:47:19 on dist-test-slave-kznw"
server_key: "833d72cdd894d4e190893d0da9b1b722"
server_key_iv: "f0d62f09794728714a8df1f3aba9d6c9"
server_key_version: "encryptionkey@0"
I20260925 02:47:19.651995 3055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790304178325258-13942-0/minicluster-data/master-0/wal/instance:
uuid: "03992beb9c554e7886160d5940498e4c"
format_stamp: "Formatted at 2026-09-25 02:47:19 on dist-test-slave-kznw"
server_key: "833d72cdd894d4e190893d0da9b1b722"
server_key_iv: "f0d62f09794728714a8df1f3aba9d6c9"
server_key_version: "encryptionkey@0"
I20260925 02:47:19.655941 3055 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260925 02:47:19.658444 3070 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:47:19.659713 3055 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260925 02:47:19.659842 3055 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790304178325258-13942-0/minicluster-data/master-0/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790304178325258-13942-0/minicluster-data/master-0/wal
uuid: "03992beb9c554e7886160d5940498e4c"
format_stamp: "Formatted at 2026-09-25 02:47:19 on dist-test-slave-kznw"
server_key: "833d72cdd894d4e190893d0da9b1b722"
server_key_iv: "f0d62f09794728714a8df1f3aba9d6c9"
server_key_version: "encryptionkey@0"
I20260925 02:47:19.659947 3055 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790304178325258-13942-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790304178325258-13942-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790304178325258-13942-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:47:19.675289 3055 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:47:19.678489 3055 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:47:19.678674 3055 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:47:19.685650 3055 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45011
I20260925 02:47:19.685720 3122 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45011 every 8 connection(s)
I20260925 02:47:19.686626 3055 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
I20260925 02:47:19.687193 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 3055
I20260925 02:47:19.687330 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790304178325258-13942-0/minicluster-data/master-0/wal/instance
I20260925 02:47:19.687639 13942 external_mini_cluster.cc:1428] Setting key a91758e7f2befecbbaa31727839b9d08
I20260925 02:47:19.689986 3123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:47:19.695519 3123 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03992beb9c554e7886160d5940498e4c: Bootstrap starting.
I20260925 02:47:19.696475 3126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:19.689058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.34:41075 (local address 127.0.0.1:45011)
0925 02:47:19.689533 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:19.689548 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:19.689585 (+ 37us) server_negotiation.cc:407] Connection header received
0925 02:47:19.690251 (+ 666us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:19.690271 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:19.690634 (+ 363us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:19.690982 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:47:19.691398 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:19.692375 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:19.693284 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:19.693779 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:19.693928 (+ 149us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:19.694784 (+ 856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:47:19.694789 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:47:19.694803 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:47:19.695227 (+ 424us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:47:19.695281 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:47:19.695304 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:47:19.695420 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:19.695602 (+ 182us) server_negotiation.cc:299] Negotiation successful
0925 02:47:19.695746 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":324,"thread_start_us":148,"threads_started":1}
I20260925 02:47:19.697930 3123 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 03992beb9c554e7886160d5940498e4c: Neither blocks nor log segments found. Creating new log.
I20260925 02:47:19.698709 3123 log.cc:824] T 00000000000000000000000000000000 P 03992beb9c554e7886160d5940498e4c: Log is configured to *not* fsync() on all Append() calls
I20260925 02:47:19.699425 13942 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 02:47:19.700599 3123 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03992beb9c554e7886160d5940498e4c: No bootstrap required, opened a new log
I20260925 02:47:19.704710 3123 raft_consensus.cc:348] T 00000000000000000000000000000000 P 03992beb9c554e7886160d5940498e4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03992beb9c554e7886160d5940498e4c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45011 } }
I20260925 02:47:19.704972 3123 raft_consensus.cc:374] T 00000000000000000000000000000000 P 03992beb9c554e7886160d5940498e4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:47:19.705039 3123 raft_consensus.cc:729] T 00000000000000000000000000000000 P 03992beb9c554e7886160d5940498e4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03992beb9c554e7886160d5940498e4c, State: Initialized, Role: FOLLOWER
I20260925 02:47:19.705399 3126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:19.701322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.34:55311 (local address 127.0.0.1:45011)
0925 02:47:19.701442 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:19.701446 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:19.701627 (+ 181us) server_negotiation.cc:407] Connection header received
0925 02:47:19.701726 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:19.701729 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:19.701773 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:19.701840 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:47:19.702074 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:19.702849 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:19.703689 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:19.704013 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:19.704119 (+ 106us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:19.704915 (+ 796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:47:19.704919 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:47:19.704921 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:47:19.704935 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:47:19.704962 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:47:19.704976 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:47:19.705066 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:19.705227 (+ 161us) server_negotiation.cc:299] Negotiation successful
0925 02:47:19.705265 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260925 02:47:19.705550 3123 consensus_queue.cc:260] T 00000000000000000000000000000000 P 03992beb9c554e7886160d5940498e4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03992beb9c554e7886160d5940498e4c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45011 } }
I20260925 02:47:19.705690 3123 raft_consensus.cc:388] T 00000000000000000000000000000000 P 03992beb9c554e7886160d5940498e4c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 02:47:19.705744 3123 raft_consensus.cc:482] T 00000000000000000000000000000000 P 03992beb9c554e7886160d5940498e4c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 02:47:19.705842 3123 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 03992beb9c554e7886160d5940498e4c [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:47:19.706694 3123 raft_consensus.cc:504] T 00000000000000000000000000000000 P 03992beb9c554e7886160d5940498e4c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03992beb9c554e7886160d5940498e4c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45011 } }
I20260925 02:47:19.707038 3123 leader_election.cc:302] T 00000000000000000000000000000000 P 03992beb9c554e7886160d5940498e4c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 03992beb9c554e7886160d5940498e4c; no voters:
I20260925 02:47:19.707325 3123 leader_election.cc:288] T 00000000000000000000000000000000 P 03992beb9c554e7886160d5940498e4c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 02:47:19.707428 3133 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 03992beb9c554e7886160d5940498e4c [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:47:19.707669 3133 raft_consensus.cc:686] T 00000000000000000000000000000000 P 03992beb9c554e7886160d5940498e4c [term 1 LEADER]: Becoming Leader. State: Replica: 03992beb9c554e7886160d5940498e4c, State: Running, Role: LEADER
I20260925 02:47:19.707974 3133 consensus_queue.cc:237] T 00000000000000000000000000000000 P 03992beb9c554e7886160d5940498e4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03992beb9c554e7886160d5940498e4c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45011 } }
I20260925 02:47:19.708463 3123 sys_catalog.cc:564] T 00000000000000000000000000000000 P 03992beb9c554e7886160d5940498e4c [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:47:19.709636 3134 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03992beb9c554e7886160d5940498e4c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "03992beb9c554e7886160d5940498e4c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03992beb9c554e7886160d5940498e4c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45011 } } }
I20260925 02:47:19.709651 3135 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03992beb9c554e7886160d5940498e4c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 03992beb9c554e7886160d5940498e4c. Latest consensus state: current_term: 1 leader_uuid: "03992beb9c554e7886160d5940498e4c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03992beb9c554e7886160d5940498e4c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45011 } } }
I20260925 02:47:19.709784 3134 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03992beb9c554e7886160d5940498e4c [sys.catalog]: This master's current role is: LEADER
I20260925 02:47:19.709786 3135 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03992beb9c554e7886160d5940498e4c [sys.catalog]: This master's current role is: LEADER
W20260925 02:47:19.713246 3148 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 03992beb9c554e7886160d5940498e4c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 02:47:19.713318 3148 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 02:47:19.713394 3143 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:47:19.714061 3143 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:47:19.719235 3143 catalog_manager.cc:1347] Generated new cluster ID: df21c1d4008643fb8d5317fcda94da03
I20260925 02:47:19.719310 3143 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 02:47:19.734107 3143 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 02:47:19.735055 3143 catalog_manager.cc:1504] Loading token signing keys...
I20260925 02:47:19.739761 3143 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 03992beb9c554e7886160d5940498e4c: Generated new TSK 0
I20260925 02:47:19.740561 3143 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 02:47:19.752681 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 3055
W20260925 02:47:19.757781 3040 connection.cc:535] client connection to 127.0.0.1:45011 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45011): Connection reset by peer (error 104)
2026-09-25T02:47:19Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1317 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790304178325258-13942-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 02:47:19 dist-test-slave-kznw krb5kdc[3154](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 02:47:19 dist-test-slave-kznw krb5kdc[3154](info): set up 2 sockets
Sep 25 02:47:19 dist-test-slave-kznw krb5kdc[3154](info): commencing operation
krb5kdc: starting...
W20260925 02:47:20.706686 13942 mini_kdc.cc:121] Time spent starting KDC: real 0.935s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790304178325258-13942-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790304178325258-13942-0/krb5kdc/test-user.keytab.
Sep 25 02:47:20 dist-test-slave-kznw krb5kdc[3154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304440, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T02:47:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T02:47:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790304178325258-13942-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790304178325258-13942-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 02:47:20.825493 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790304178325258-13942-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790304178325258-13942-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790304178325258-13942-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42689
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790304178325258-13942-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42689
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:47:20.935899 3167 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:47:20.936178 3167 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:47:20.936220 3167 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:47:20.940254 3167 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:47:20.940315 3167 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:47:20.940333 3167 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:47:20.940349 3167 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:47:20.940363 3167 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:47:20.945617 3167 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790304178325258-13942-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790304178325258-13942-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42689
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790304178325258-13942-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42689
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3167
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790304178325258-13942-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:47:20.946672 3167 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:47:20.947711 3167 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:47:20.956007 3172 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:47:20.955989 3173 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:47:20.956140 3167 server_base.cc:1034] running on GCE node
W20260925 02:47:20.955972 3175 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:47:20.956704 3167 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:47:20.957826 3167 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:47:20.958968 3167 hybrid_clock.cc:648] HybridClock initialized: now 1790304440958945 us; error 33 us; skew 500 ppm
Sep 25 02:47:20 dist-test-slave-kznw krb5kdc[3154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304440, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:47:20.961645 3167 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 02:47:20.962514 3167 webserver.cc:460] Webserver started at http://127.0.0.1:36125/ using document root <none> and password file <none>
I20260925 02:47:20.963078 3167 fs_manager.cc:359] Metadata directory not provided
I20260925 02:47:20.963146 3167 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:47:20.963342 3167 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:47:20.965318 3167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790304178325258-13942-0/minicluster-data/master-0/data/instance:
uuid: "3e4aeb496be84eb4a878eb9f101ed761"
format_stamp: "Formatted at 2026-09-25 02:47:20 on dist-test-slave-kznw"
server_key: "e841b20098faa8839cfea5ea49b76a36"
server_key_iv: "bb5c5961d024597fd499c71bb00093f9"
server_key_version: "encryptionkey@0"
I20260925 02:47:20.965862 3167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790304178325258-13942-0/minicluster-data/master-0/wal/instance:
uuid: "3e4aeb496be84eb4a878eb9f101ed761"
format_stamp: "Formatted at 2026-09-25 02:47:20 on dist-test-slave-kznw"
server_key: "e841b20098faa8839cfea5ea49b76a36"
server_key_iv: "bb5c5961d024597fd499c71bb00093f9"
server_key_version: "encryptionkey@0"
I20260925 02:47:20.969462 3167 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260925 02:47:20.971983 3182 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:47:20.973266 3167 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260925 02:47:20.973379 3167 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790304178325258-13942-0/minicluster-data/master-0/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790304178325258-13942-0/minicluster-data/master-0/wal
uuid: "3e4aeb496be84eb4a878eb9f101ed761"
format_stamp: "Formatted at 2026-09-25 02:47:20 on dist-test-slave-kznw"
server_key: "e841b20098faa8839cfea5ea49b76a36"
server_key_iv: "bb5c5961d024597fd499c71bb00093f9"
server_key_version: "encryptionkey@0"
I20260925 02:47:20.973551 3167 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790304178325258-13942-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790304178325258-13942-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790304178325258-13942-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:47:20.977926 3167 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:47:20.980887 3167 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:47:20.981027 3167 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:47:20.987613 3167 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42689
I20260925 02:47:20.987627 3234 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42689 every 8 connection(s)
I20260925 02:47:20.988474 3167 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
I20260925 02:47:20.991220 3235 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:47:20.991657 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 3167
I20260925 02:47:20.991782 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790304178325258-13942-0/minicluster-data/master-0/wal/instance
I20260925 02:47:20.992098 13942 external_mini_cluster.cc:1428] Setting key c26b982ab2d082a9b6d48fc0639d401c
I20260925 02:47:20.997051 3238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:20.993500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.34:55409 (local address 127.0.0.1:42689)
0925 02:47:20.993905 (+ 405us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:20.993919 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:20.993952 (+ 33us) server_negotiation.cc:407] Connection header received
0925 02:47:20.994616 (+ 664us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:20.994635 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:20.994934 (+ 299us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:20.995243 (+ 309us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:47:20.995955 (+ 712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:47:20.995960 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:47:20.995972 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:47:20.995982 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:47:20.996036 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:47:20.996054 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:47:20.996173 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:20.996239 (+ 66us) server_negotiation.cc:299] Negotiation successful
0925 02:47:20.996390 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":268,"thread_start_us":114,"threads_started":1}
I20260925 02:47:20.997777 3235 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e4aeb496be84eb4a878eb9f101ed761: Bootstrap starting.
I20260925 02:47:20.999578 13942 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 02:47:21.000028 3235 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3e4aeb496be84eb4a878eb9f101ed761: Neither blocks nor log segments found. Creating new log.
I20260925 02:47:21.000734 3235 log.cc:824] T 00000000000000000000000000000000 P 3e4aeb496be84eb4a878eb9f101ed761: Log is configured to *not* fsync() on all Append() calls
I20260925 02:47:21.002573 3235 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e4aeb496be84eb4a878eb9f101ed761: No bootstrap required, opened a new log
I20260925 02:47:21.002813 3238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:21.001384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.34:37053 (local address 127.0.0.1:42689)
0925 02:47:21.001486 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:21.001491 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:21.001658 (+ 167us) server_negotiation.cc:407] Connection header received
0925 02:47:21.001769 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:21.001772 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:21.001803 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:21.001865 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:47:21.002472 (+ 607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:47:21.002476 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:47:21.002479 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:47:21.002490 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:47:21.002518 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:47:21.002528 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:47:21.002597 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:21.002669 (+ 72us) server_negotiation.cc:299] Negotiation successful
0925 02:47:21.002706 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260925 02:47:21.005999 3235 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3e4aeb496be84eb4a878eb9f101ed761 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e4aeb496be84eb4a878eb9f101ed761" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42689 } }
I20260925 02:47:21.006209 3235 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3e4aeb496be84eb4a878eb9f101ed761 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:47:21.006248 3235 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3e4aeb496be84eb4a878eb9f101ed761 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e4aeb496be84eb4a878eb9f101ed761, State: Initialized, Role: FOLLOWER
I20260925 02:47:21.006642 3235 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3e4aeb496be84eb4a878eb9f101ed761 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e4aeb496be84eb4a878eb9f101ed761" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42689 } }
I20260925 02:47:21.006769 3235 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3e4aeb496be84eb4a878eb9f101ed761 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 02:47:21.006827 3235 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3e4aeb496be84eb4a878eb9f101ed761 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 02:47:21.006918 3235 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3e4aeb496be84eb4a878eb9f101ed761 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:47:21.007727 3235 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3e4aeb496be84eb4a878eb9f101ed761 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e4aeb496be84eb4a878eb9f101ed761" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42689 } }
I20260925 02:47:21.008049 3235 leader_election.cc:302] T 00000000000000000000000000000000 P 3e4aeb496be84eb4a878eb9f101ed761 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3e4aeb496be84eb4a878eb9f101ed761; no voters:
I20260925 02:47:21.008356 3235 leader_election.cc:288] T 00000000000000000000000000000000 P 3e4aeb496be84eb4a878eb9f101ed761 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 02:47:21.008450 3245 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3e4aeb496be84eb4a878eb9f101ed761 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:47:21.008713 3245 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3e4aeb496be84eb4a878eb9f101ed761 [term 1 LEADER]: Becoming Leader. State: Replica: 3e4aeb496be84eb4a878eb9f101ed761, State: Running, Role: LEADER
I20260925 02:47:21.009019 3245 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3e4aeb496be84eb4a878eb9f101ed761 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e4aeb496be84eb4a878eb9f101ed761" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42689 } }
I20260925 02:47:21.009511 3235 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3e4aeb496be84eb4a878eb9f101ed761 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:47:21.010509 3247 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e4aeb496be84eb4a878eb9f101ed761 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3e4aeb496be84eb4a878eb9f101ed761" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e4aeb496be84eb4a878eb9f101ed761" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42689 } } }
I20260925 02:47:21.010633 3247 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e4aeb496be84eb4a878eb9f101ed761 [sys.catalog]: This master's current role is: LEADER
I20260925 02:47:21.010946 3246 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e4aeb496be84eb4a878eb9f101ed761 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3e4aeb496be84eb4a878eb9f101ed761. Latest consensus state: current_term: 1 leader_uuid: "3e4aeb496be84eb4a878eb9f101ed761" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e4aeb496be84eb4a878eb9f101ed761" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42689 } } }
I20260925 02:47:21.011037 3254 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:47:21.011052 3246 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e4aeb496be84eb4a878eb9f101ed761 [sys.catalog]: This master's current role is: LEADER
I20260925 02:47:21.013813 3254 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:47:21.019181 3254 catalog_manager.cc:1347] Generated new cluster ID: 7fbc687d8d444822b2f57b4069ddeab4
I20260925 02:47:21.019241 3254 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 02:47:21.052601 3254 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 02:47:21.052925 3254 catalog_manager.cc:1504] Loading token signing keys...
I20260925 02:47:21.062835 3254 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3e4aeb496be84eb4a878eb9f101ed761: Generated new TSK 0
I20260925 02:47:21.063654 3254 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 02:47:21.073613 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 3167
2026-09-25T02:47:21Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1321 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790304178325258-13942-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 02:47:21 dist-test-slave-kznw krb5kdc[3266](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 02:47:21 dist-test-slave-kznw krb5kdc[3266](info): set up 2 sockets
Sep 25 02:47:21 dist-test-slave-kznw krb5kdc[3266](info): commencing operation
krb5kdc: starting...
W20260925 02:47:22.038520 13942 mini_kdc.cc:121] Time spent starting KDC: real 0.946s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790304178325258-13942-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790304178325258-13942-0/krb5kdc/test-user.keytab.
Sep 25 02:47:22 dist-test-slave-kznw krb5kdc[3266](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304442, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T02:47:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T02:47:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.157.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.157.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.157.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab.
Entry for principal kudu/127.13.157.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab.
I20260925 02:47:22.154603 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790304178325258-13942-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790304178325258-13942-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790304178325258-13942-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.157.190:41053
--webserver_interface=127.13.157.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab
--principal=kudu/127.13.157.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:38163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.157.190:41053
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:47:22.264873 3279 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:47:22.265125 3279 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:47:22.265162 3279 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:47:22.269096 3279 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:47:22.269155 3279 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:47:22.269173 3279 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:47:22.269188 3279 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:47:22.269201 3279 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:47:22.274335 3279 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:38163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790304178325258-13942-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790304178325258-13942-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.157.190:41053
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab
--principal=kudu/127.13.157.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.157.190:41053
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.157.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3279
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790304178325258-13942-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:47:22.275429 3279 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:47:22.276471 3279 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:47:22.284653 3287 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:47:22.284744 3284 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:47:22.284664 3285 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:47:22.284807 3279 server_base.cc:1034] running on GCE node
I20260925 02:47:22.285445 3279 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:47:22.286594 3279 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:47:22.287766 3279 hybrid_clock.cc:648] HybridClock initialized: now 1790304442287735 us; error 44 us; skew 500 ppm
Sep 25 02:47:22 dist-test-slave-kznw krb5kdc[3266](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304442, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:47:22.296837 3279 init.cc:377] Logged in from keytab as kudu/127.13.157.190@KRBTEST.COM (short username kudu)
I20260925 02:47:22.297772 3279 webserver.cc:460] Webserver started at http://127.13.157.190:41087/ using document root <none> and password file <none>
I20260925 02:47:22.298301 3279 fs_manager.cc:359] Metadata directory not provided
I20260925 02:47:22.298358 3279 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:47:22.298543 3279 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:47:22.300647 3279 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790304178325258-13942-0/minicluster-data/master-0/data/instance:
uuid: "d330474f877649d69cc6ede89ff0acb2"
format_stamp: "Formatted at 2026-09-25 02:47:22 on dist-test-slave-kznw"
server_key: "57e65dba2a6788d8c4f99c396c051ea4"
server_key_iv: "341c1ff7322f9af4fd83aef2b3dde273"
server_key_version: "encryptionkey@0"
I20260925 02:47:22.301165 3279 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790304178325258-13942-0/minicluster-data/master-0/wal/instance:
uuid: "d330474f877649d69cc6ede89ff0acb2"
format_stamp: "Formatted at 2026-09-25 02:47:22 on dist-test-slave-kznw"
server_key: "57e65dba2a6788d8c4f99c396c051ea4"
server_key_iv: "341c1ff7322f9af4fd83aef2b3dde273"
server_key_version: "encryptionkey@0"
I20260925 02:47:22.304807 3279 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.005s
I20260925 02:47:22.307099 3294 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:47:22.308306 3279 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.002s
I20260925 02:47:22.308408 3279 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790304178325258-13942-0/minicluster-data/master-0/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790304178325258-13942-0/minicluster-data/master-0/wal
uuid: "d330474f877649d69cc6ede89ff0acb2"
format_stamp: "Formatted at 2026-09-25 02:47:22 on dist-test-slave-kznw"
server_key: "57e65dba2a6788d8c4f99c396c051ea4"
server_key_iv: "341c1ff7322f9af4fd83aef2b3dde273"
server_key_version: "encryptionkey@0"
I20260925 02:47:22.308490 3279 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790304178325258-13942-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790304178325258-13942-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790304178325258-13942-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:47:22.340363 3279 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:47:22.343474 3279 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:47:22.343628 3279 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:47:22.350339 3279 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.190:41053
I20260925 02:47:22.350383 3347 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.190:41053 every 8 connection(s)
I20260925 02:47:22.351281 3279 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
I20260925 02:47:22.353991 3348 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:47:22.359400 3348 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d330474f877649d69cc6ede89ff0acb2: Bootstrap starting.
I20260925 02:47:22.361212 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 3279
I20260925 02:47:22.361344 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790304178325258-13942-0/minicluster-data/master-0/wal/instance
I20260925 02:47:22.361642 13942 external_mini_cluster.cc:1428] Setting key 7dcc7790004da2f2eed3b613462f348e
I20260925 02:47:22.362074 3348 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d330474f877649d69cc6ede89ff0acb2: Neither blocks nor log segments found. Creating new log.
I20260925 02:47:22.362864 3348 log.cc:824] T 00000000000000000000000000000000 P d330474f877649d69cc6ede89ff0acb2: Log is configured to *not* fsync() on all Append() calls
I20260925 02:47:22.364814 3348 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d330474f877649d69cc6ede89ff0acb2: No bootstrap required, opened a new log
I20260925 02:47:22.368749 3348 raft_consensus.cc:348] T 00000000000000000000000000000000 P d330474f877649d69cc6ede89ff0acb2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d330474f877649d69cc6ede89ff0acb2" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 41053 } }
I20260925 02:47:22.369010 3348 raft_consensus.cc:374] T 00000000000000000000000000000000 P d330474f877649d69cc6ede89ff0acb2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:47:22.369104 3348 raft_consensus.cc:729] T 00000000000000000000000000000000 P d330474f877649d69cc6ede89ff0acb2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d330474f877649d69cc6ede89ff0acb2, State: Initialized, Role: FOLLOWER
I20260925 02:47:22.369627 3348 consensus_queue.cc:260] T 00000000000000000000000000000000 P d330474f877649d69cc6ede89ff0acb2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d330474f877649d69cc6ede89ff0acb2" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 41053 } }
I20260925 02:47:22.369804 3348 raft_consensus.cc:388] T 00000000000000000000000000000000 P d330474f877649d69cc6ede89ff0acb2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 02:47:22.369872 3348 raft_consensus.cc:482] T 00000000000000000000000000000000 P d330474f877649d69cc6ede89ff0acb2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 02:47:22.369980 3348 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d330474f877649d69cc6ede89ff0acb2 [term 0 FOLLOWER]: Advancing to term 1
Sep 25 02:47:22 dist-test-slave-kznw krb5kdc[3266](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304442, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.157.190@KRBTEST.COM
I20260925 02:47:22.370890 3348 raft_consensus.cc:504] T 00000000000000000000000000000000 P d330474f877649d69cc6ede89ff0acb2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d330474f877649d69cc6ede89ff0acb2" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 41053 } }
I20260925 02:47:22.371260 3348 leader_election.cc:302] T 00000000000000000000000000000000 P d330474f877649d69cc6ede89ff0acb2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d330474f877649d69cc6ede89ff0acb2; no voters:
I20260925 02:47:22.371584 3348 leader_election.cc:288] T 00000000000000000000000000000000 P d330474f877649d69cc6ede89ff0acb2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 02:47:22.371647 3353 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d330474f877649d69cc6ede89ff0acb2 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:47:22.371850 3353 raft_consensus.cc:686] T 00000000000000000000000000000000 P d330474f877649d69cc6ede89ff0acb2 [term 1 LEADER]: Becoming Leader. State: Replica: d330474f877649d69cc6ede89ff0acb2, State: Running, Role: LEADER
I20260925 02:47:22.372159 3353 consensus_queue.cc:237] T 00000000000000000000000000000000 P d330474f877649d69cc6ede89ff0acb2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d330474f877649d69cc6ede89ff0acb2" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 41053 } }
I20260925 02:47:22.372910 3348 sys_catalog.cc:564] T 00000000000000000000000000000000 P d330474f877649d69cc6ede89ff0acb2 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:47:22.373374 3354 sys_catalog.cc:455] T 00000000000000000000000000000000 P d330474f877649d69cc6ede89ff0acb2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d330474f877649d69cc6ede89ff0acb2. Latest consensus state: current_term: 1 leader_uuid: "d330474f877649d69cc6ede89ff0acb2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d330474f877649d69cc6ede89ff0acb2" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 41053 } } }
I20260925 02:47:22.373395 3355 sys_catalog.cc:455] T 00000000000000000000000000000000 P d330474f877649d69cc6ede89ff0acb2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d330474f877649d69cc6ede89ff0acb2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d330474f877649d69cc6ede89ff0acb2" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 41053 } } }
I20260925 02:47:22.373574 3355 sys_catalog.cc:458] T 00000000000000000000000000000000 P d330474f877649d69cc6ede89ff0acb2 [sys.catalog]: This master's current role is: LEADER
I20260925 02:47:22.373867 3354 sys_catalog.cc:458] T 00000000000000000000000000000000 P d330474f877649d69cc6ede89ff0acb2 [sys.catalog]: This master's current role is: LEADER
I20260925 02:47:22.374202 3359 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:47:22.376765 3351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:22.362892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53959 (local address 127.13.157.190:41053)
0925 02:47:22.363376 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:22.363393 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:22.363428 (+ 35us) server_negotiation.cc:407] Connection header received
0925 02:47:22.364102 (+ 674us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:22.364121 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:22.364457 (+ 336us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:22.364728 (+ 271us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:47:22.365608 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:22.366597 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:22.367837 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:22.368316 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:22.368438 (+ 122us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:22.371332 (+ 2894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:47:22.371337 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:47:22.371352 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:47:22.371383 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:47:22.373081 (+ 1698us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:47:22.373675 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:47:22.373682 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:47:22.373689 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:47:22.373835 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:47:22.374246 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:47:22.374250 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:47:22.374252 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:47:22.374751 (+ 499us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:47:22.374987 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:22.375514 (+ 527us) server_negotiation.cc:299] Negotiation successful
0925 02:47:22.375741 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":125,"threads_started":1}
I20260925 02:47:22.377473 3359 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:47:22.383490 3359 catalog_manager.cc:1347] Generated new cluster ID: cd4288085bbd434b950a4fb4ec0c7036
I20260925 02:47:22.383545 3359 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 02:47:22.387274 13942 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 02:47:22.399899 3351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:22.391400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56933 (local address 127.13.157.190:41053)
0925 02:47:22.391523 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:22.391527 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:22.391716 (+ 189us) server_negotiation.cc:407] Connection header received
0925 02:47:22.391846 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:22.391849 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:22.391898 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:22.391974 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:47:22.392686 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:22.393550 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:22.394609 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:22.394991 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:22.395092 (+ 101us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:22.396596 (+ 1504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:47:22.396600 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:47:22.396602 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:47:22.396629 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:47:22.398168 (+ 1539us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:47:22.398643 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:47:22.398646 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:47:22.398648 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:47:22.398733 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:47:22.399024 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:47:22.399027 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:47:22.399028 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:47:22.399253 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:47:22.399399 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:22.399625 (+ 226us) server_negotiation.cc:299] Negotiation successful
0925 02:47:22.399725 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":40}
I20260925 02:47:22.412742 3359 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 02:47:22.413703 3359 catalog_manager.cc:1504] Loading token signing keys...
I20260925 02:47:22.420331 3359 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d330474f877649d69cc6ede89ff0acb2: Generated new TSK 0
I20260925 02:47:22.421020 3359 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 02:47:22.426057 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 3279
W20260925 02:47:22.431133 3264 connection.cc:535] client connection to 127.13.157.190:41053 recv error: Network error: failed to read from TLS socket (remote: 127.13.157.190:41053): Connection reset by peer (error 104)
2026-09-25T02:47:22Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1352 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790304178325258-13942-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 02:47:22 dist-test-slave-kznw krb5kdc[3379](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 02:47:22 dist-test-slave-kznw krb5kdc[3379](info): set up 2 sockets
Sep 25 02:47:22 dist-test-slave-kznw krb5kdc[3379](info): commencing operation
krb5kdc: starting...
W20260925 02:47:23.386507 13942 mini_kdc.cc:121] Time spent starting KDC: real 0.941s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790304178325258-13942-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790304178325258-13942-0/krb5kdc/test-user.keytab.
Sep 25 02:47:23 dist-test-slave-kznw krb5kdc[3379](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304443, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T02:47:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T02:47:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.157.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.157.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.157.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab.
Entry for principal kudu/127.13.157.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab.
I20260925 02:47:23.501972 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790304178325258-13942-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790304178325258-13942-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790304178325258-13942-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.157.190:38241
--webserver_interface=127.13.157.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab
--principal=kudu/127.13.157.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:34249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.157.190:38241
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:47:23.609025 3392 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:47:23.609305 3392 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:47:23.609365 3392 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:47:23.613394 3392 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:47:23.613483 3392 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:47:23.613511 3392 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:47:23.613533 3392 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:47:23.613559 3392 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:47:23.619275 3392 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:34249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790304178325258-13942-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790304178325258-13942-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.157.190:38241
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab
--principal=kudu/127.13.157.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.157.190:38241
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.157.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3392
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790304178325258-13942-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:47:23.620412 3392 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:47:23.621461 3392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:47:23.629664 3398 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:47:23.629719 3400 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:47:23.629664 3397 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:47:23.629777 3392 server_base.cc:1034] running on GCE node
I20260925 02:47:23.630556 3392 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:47:23.631668 3392 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:47:23.632823 3392 hybrid_clock.cc:648] HybridClock initialized: now 1790304443632807 us; error 28 us; skew 500 ppm
Sep 25 02:47:23 dist-test-slave-kznw krb5kdc[3379](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304443, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:47:23.635423 3392 init.cc:377] Logged in from keytab as kudu/127.13.157.190@KRBTEST.COM (short username kudu)
I20260925 02:47:23.636349 3392 webserver.cc:460] Webserver started at http://127.13.157.190:43863/ using document root <none> and password file <none>
I20260925 02:47:23.637018 3392 fs_manager.cc:359] Metadata directory not provided
I20260925 02:47:23.637094 3392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:47:23.637305 3392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:47:23.639952 3392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790304178325258-13942-0/minicluster-data/master-0/data/instance:
uuid: "5293d6624bc24373b6dda5b037700eb3"
format_stamp: "Formatted at 2026-09-25 02:47:23 on dist-test-slave-kznw"
server_key: "c32851f68eec3e21919082fc4dd8d2d3"
server_key_iv: "d6f2252091d1d43bfad74231f9db76fe"
server_key_version: "encryptionkey@0"
I20260925 02:47:23.640532 3392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790304178325258-13942-0/minicluster-data/master-0/wal/instance:
uuid: "5293d6624bc24373b6dda5b037700eb3"
format_stamp: "Formatted at 2026-09-25 02:47:23 on dist-test-slave-kznw"
server_key: "c32851f68eec3e21919082fc4dd8d2d3"
server_key_iv: "d6f2252091d1d43bfad74231f9db76fe"
server_key_version: "encryptionkey@0"
I20260925 02:47:23.644273 3392 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260925 02:47:23.646617 3407 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:47:23.648003 3392 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260925 02:47:23.648119 3392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790304178325258-13942-0/minicluster-data/master-0/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790304178325258-13942-0/minicluster-data/master-0/wal
uuid: "5293d6624bc24373b6dda5b037700eb3"
format_stamp: "Formatted at 2026-09-25 02:47:23 on dist-test-slave-kznw"
server_key: "c32851f68eec3e21919082fc4dd8d2d3"
server_key_iv: "d6f2252091d1d43bfad74231f9db76fe"
server_key_version: "encryptionkey@0"
I20260925 02:47:23.648238 3392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790304178325258-13942-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790304178325258-13942-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790304178325258-13942-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:47:23.665062 3392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:47:23.668064 3392 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:47:23.668254 3392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:47:23.675535 3392 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.190:38241
I20260925 02:47:23.675648 3459 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.190:38241 every 8 connection(s)
I20260925 02:47:23.676541 3392 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
I20260925 02:47:23.677639 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 3392
I20260925 02:47:23.677758 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790304178325258-13942-0/minicluster-data/master-0/wal/instance
I20260925 02:47:23.678056 13942 external_mini_cluster.cc:1428] Setting key e9027bdca4c6140bbbbaa8d667f2f8f9
I20260925 02:47:23.680568 3460 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:47:23.686192 3460 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5293d6624bc24373b6dda5b037700eb3: Bootstrap starting.
I20260925 02:47:23.686758 3463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:23.679445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32963 (local address 127.13.157.190:38241)
0925 02:47:23.679849 (+ 404us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:23.679865 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:23.679902 (+ 37us) server_negotiation.cc:407] Connection header received
0925 02:47:23.680571 (+ 669us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:23.680588 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:23.680973 (+ 385us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:23.681288 (+ 315us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:47:23.681722 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:23.682705 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:23.683629 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:23.684098 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:23.684244 (+ 146us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:23.685123 (+ 879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:47:23.685128 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:47:23.685139 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:47:23.685526 (+ 387us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:47:23.685580 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:47:23.685603 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:47:23.685718 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:23.685907 (+ 189us) server_negotiation.cc:299] Negotiation successful
0925 02:47:23.686042 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":268,"thread_start_us":122,"threads_started":1}
I20260925 02:47:23.688577 3460 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5293d6624bc24373b6dda5b037700eb3: Neither blocks nor log segments found. Creating new log.
I20260925 02:47:23.689366 3460 log.cc:824] T 00000000000000000000000000000000 P 5293d6624bc24373b6dda5b037700eb3: Log is configured to *not* fsync() on all Append() calls
I20260925 02:47:23.689788 13942 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 02:47:23.691604 3460 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5293d6624bc24373b6dda5b037700eb3: No bootstrap required, opened a new log
I20260925 02:47:23.695871 3463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:23.691933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44091 (local address 127.13.157.190:38241)
0925 02:47:23.692071 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:23.692075 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:23.692249 (+ 174us) server_negotiation.cc:407] Connection header received
0925 02:47:23.692326 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:23.692329 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:23.692371 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:23.692464 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:47:23.692672 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:23.693429 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:23.694231 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:23.694559 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:23.694663 (+ 104us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:23.695421 (+ 758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:47:23.695425 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:47:23.695429 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:47:23.695444 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:47:23.695472 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:47:23.695485 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:47:23.695578 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:23.695706 (+ 128us) server_negotiation.cc:299] Negotiation successful
0925 02:47:23.695743 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260925 02:47:23.696044 3460 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5293d6624bc24373b6dda5b037700eb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5293d6624bc24373b6dda5b037700eb3" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 38241 } }
I20260925 02:47:23.696282 3460 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5293d6624bc24373b6dda5b037700eb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:47:23.696341 3460 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5293d6624bc24373b6dda5b037700eb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5293d6624bc24373b6dda5b037700eb3, State: Initialized, Role: FOLLOWER
I20260925 02:47:23.696800 3460 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5293d6624bc24373b6dda5b037700eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5293d6624bc24373b6dda5b037700eb3" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 38241 } }
I20260925 02:47:23.696943 3460 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5293d6624bc24373b6dda5b037700eb3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 02:47:23.697003 3460 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5293d6624bc24373b6dda5b037700eb3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 02:47:23.697098 3460 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5293d6624bc24373b6dda5b037700eb3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:47:23.697983 3460 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5293d6624bc24373b6dda5b037700eb3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5293d6624bc24373b6dda5b037700eb3" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 38241 } }
I20260925 02:47:23.698298 3460 leader_election.cc:302] T 00000000000000000000000000000000 P 5293d6624bc24373b6dda5b037700eb3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5293d6624bc24373b6dda5b037700eb3; no voters:
I20260925 02:47:23.698575 3460 leader_election.cc:288] T 00000000000000000000000000000000 P 5293d6624bc24373b6dda5b037700eb3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 02:47:23.698697 3470 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5293d6624bc24373b6dda5b037700eb3 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:47:23.698884 3470 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5293d6624bc24373b6dda5b037700eb3 [term 1 LEADER]: Becoming Leader. State: Replica: 5293d6624bc24373b6dda5b037700eb3, State: Running, Role: LEADER
I20260925 02:47:23.699297 3470 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5293d6624bc24373b6dda5b037700eb3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5293d6624bc24373b6dda5b037700eb3" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 38241 } }
I20260925 02:47:23.699806 3460 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5293d6624bc24373b6dda5b037700eb3 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:47:23.700932 3471 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5293d6624bc24373b6dda5b037700eb3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5293d6624bc24373b6dda5b037700eb3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5293d6624bc24373b6dda5b037700eb3" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 38241 } } }
I20260925 02:47:23.701069 3471 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5293d6624bc24373b6dda5b037700eb3 [sys.catalog]: This master's current role is: LEADER
I20260925 02:47:23.701416 3472 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5293d6624bc24373b6dda5b037700eb3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5293d6624bc24373b6dda5b037700eb3. Latest consensus state: current_term: 1 leader_uuid: "5293d6624bc24373b6dda5b037700eb3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5293d6624bc24373b6dda5b037700eb3" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 38241 } } }
I20260925 02:47:23.701509 3472 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5293d6624bc24373b6dda5b037700eb3 [sys.catalog]: This master's current role is: LEADER
W20260925 02:47:23.704027 3485 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5293d6624bc24373b6dda5b037700eb3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 02:47:23.704094 3485 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 02:47:23.704169 3481 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:47:23.704864 3481 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:47:23.710029 3481 catalog_manager.cc:1347] Generated new cluster ID: c64361b8b3b141349f7291230d2acabb
I20260925 02:47:23.710100 3481 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 02:47:23.723942 3481 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 02:47:23.724874 3481 catalog_manager.cc:1504] Loading token signing keys...
I20260925 02:47:23.730715 3481 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5293d6624bc24373b6dda5b037700eb3: Generated new TSK 0
I20260925 02:47:23.731418 3481 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 02:47:23.739931 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 3392
W20260925 02:47:23.745559 3377 connection.cc:535] client connection to 127.13.157.190:38241 recv error: Network error: failed to read from TLS socket (remote: 127.13.157.190:38241): Connection reset by peer (error 104)
2026-09-25T02:47:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1314 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790304178325258-13942-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 02:47:23 dist-test-slave-kznw krb5kdc[3491](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 02:47:23 dist-test-slave-kznw krb5kdc[3491](info): set up 2 sockets
Sep 25 02:47:23 dist-test-slave-kznw krb5kdc[3491](info): commencing operation
krb5kdc: starting...
W20260925 02:47:24.694536 13942 mini_kdc.cc:121] Time spent starting KDC: real 0.935s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790304178325258-13942-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790304178325258-13942-0/krb5kdc/test-user.keytab.
Sep 25 02:47:24 dist-test-slave-kznw krb5kdc[3491](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304444, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T02:47:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T02:47:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.157.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.157.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.157.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab.
Entry for principal kudu/127.13.157.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab.
I20260925 02:47:24.813476 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790304178325258-13942-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790304178325258-13942-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790304178325258-13942-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.157.190:36527
--webserver_interface=127.13.157.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab
--principal=kudu/127.13.157.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:40491
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.157.190:36527
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:47:24.922829 3504 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:47:24.923148 3504 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:47:24.923213 3504 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:47:24.927121 3504 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:47:24.927179 3504 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:47:24.927206 3504 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:47:24.927227 3504 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:47:24.927256 3504 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:47:24.932559 3504 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:40491
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790304178325258-13942-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790304178325258-13942-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.157.190:36527
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab
--principal=kudu/127.13.157.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.157.190:36527
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.157.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3504
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790304178325258-13942-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:47:24.933590 3504 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:47:24.934600 3504 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:47:24.942873 3510 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:47:24.942880 3509 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:47:24.942982 3512 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:47:24.943135 3504 server_base.cc:1034] running on GCE node
I20260925 02:47:24.943751 3504 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:47:24.944885 3504 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:47:24.946060 3504 hybrid_clock.cc:648] HybridClock initialized: now 1790304444946044 us; error 29 us; skew 500 ppm
Sep 25 02:47:24 dist-test-slave-kznw krb5kdc[3491](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304444, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:47:24.948843 3504 init.cc:377] Logged in from keytab as kudu/127.13.157.190@KRBTEST.COM (short username kudu)
I20260925 02:47:24.949762 3504 webserver.cc:460] Webserver started at http://127.13.157.190:42827/ using document root <none> and password file <none>
I20260925 02:47:24.950304 3504 fs_manager.cc:359] Metadata directory not provided
I20260925 02:47:24.950371 3504 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:47:24.950577 3504 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:47:24.952709 3504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790304178325258-13942-0/minicluster-data/master-0/data/instance:
uuid: "a7af291ee1c443908e5c7a88a871b896"
format_stamp: "Formatted at 2026-09-25 02:47:24 on dist-test-slave-kznw"
server_key: "3579f02752ffbd9f2adc38db3a72f000"
server_key_iv: "78c0fc0e5e6b2d6d62c1238aea594ab0"
server_key_version: "encryptionkey@0"
I20260925 02:47:24.953279 3504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790304178325258-13942-0/minicluster-data/master-0/wal/instance:
uuid: "a7af291ee1c443908e5c7a88a871b896"
format_stamp: "Formatted at 2026-09-25 02:47:24 on dist-test-slave-kznw"
server_key: "3579f02752ffbd9f2adc38db3a72f000"
server_key_iv: "78c0fc0e5e6b2d6d62c1238aea594ab0"
server_key_version: "encryptionkey@0"
I20260925 02:47:24.957068 3504 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260925 02:47:24.959532 3519 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:47:24.960747 3504 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260925 02:47:24.960862 3504 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790304178325258-13942-0/minicluster-data/master-0/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790304178325258-13942-0/minicluster-data/master-0/wal
uuid: "a7af291ee1c443908e5c7a88a871b896"
format_stamp: "Formatted at 2026-09-25 02:47:24 on dist-test-slave-kznw"
server_key: "3579f02752ffbd9f2adc38db3a72f000"
server_key_iv: "78c0fc0e5e6b2d6d62c1238aea594ab0"
server_key_version: "encryptionkey@0"
I20260925 02:47:24.960983 3504 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790304178325258-13942-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790304178325258-13942-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790304178325258-13942-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:47:24.965426 3504 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:47:24.968197 3504 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:47:24.968355 3504 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:47:24.974793 3504 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.190:36527
I20260925 02:47:24.974833 3571 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.190:36527 every 8 connection(s)
I20260925 02:47:24.975713 3504 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
I20260925 02:47:24.978417 3572 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:47:24.979460 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 3504
I20260925 02:47:24.979566 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790304178325258-13942-0/minicluster-data/master-0/wal/instance
I20260925 02:47:24.979856 13942 external_mini_cluster.cc:1428] Setting key 1f53da0d78d597b500f612f11058da2a
I20260925 02:47:24.985268 3572 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7af291ee1c443908e5c7a88a871b896: Bootstrap starting.
I20260925 02:47:24.985936 3575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:24.981626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41903 (local address 127.13.157.190:36527)
0925 02:47:24.982069 (+ 443us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:24.982089 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:24.982131 (+ 42us) server_negotiation.cc:407] Connection header received
0925 02:47:24.983017 (+ 886us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:24.983059 (+ 42us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:24.983444 (+ 385us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:24.983804 (+ 360us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:47:24.984523 (+ 719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:47:24.984529 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:47:24.984544 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:47:24.984555 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:47:24.984617 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:47:24.984641 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:47:24.984774 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:24.984836 (+ 62us) server_negotiation.cc:299] Negotiation successful
0925 02:47:24.984986 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":113,"threads_started":1}
I20260925 02:47:24.987771 3572 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a7af291ee1c443908e5c7a88a871b896: Neither blocks nor log segments found. Creating new log.
I20260925 02:47:24.988514 3572 log.cc:824] T 00000000000000000000000000000000 P a7af291ee1c443908e5c7a88a871b896: Log is configured to *not* fsync() on all Append() calls
I20260925 02:47:24.988754 13942 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 02:47:24.990473 3572 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7af291ee1c443908e5c7a88a871b896: No bootstrap required, opened a new log
I20260925 02:47:24.992189 3575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:24.990713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59167 (local address 127.13.157.190:36527)
0925 02:47:24.990825 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:24.990829 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:24.991069 (+ 240us) server_negotiation.cc:407] Connection header received
0925 02:47:24.991153 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:24.991156 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:24.991189 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:24.991265 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:47:24.991852 (+ 587us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:47:24.991856 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:47:24.991859 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:47:24.991870 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:47:24.991897 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:47:24.991909 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:47:24.991997 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:24.992046 (+ 49us) server_negotiation.cc:299] Negotiation successful
0925 02:47:24.992081 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260925 02:47:24.994277 3572 raft_consensus.cc:348] T 00000000000000000000000000000000 P a7af291ee1c443908e5c7a88a871b896 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7af291ee1c443908e5c7a88a871b896" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 36527 } }
I20260925 02:47:24.994496 3572 raft_consensus.cc:374] T 00000000000000000000000000000000 P a7af291ee1c443908e5c7a88a871b896 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:47:24.994537 3572 raft_consensus.cc:729] T 00000000000000000000000000000000 P a7af291ee1c443908e5c7a88a871b896 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7af291ee1c443908e5c7a88a871b896, State: Initialized, Role: FOLLOWER
I20260925 02:47:24.994922 3572 consensus_queue.cc:260] T 00000000000000000000000000000000 P a7af291ee1c443908e5c7a88a871b896 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7af291ee1c443908e5c7a88a871b896" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 36527 } }
I20260925 02:47:24.995054 3572 raft_consensus.cc:388] T 00000000000000000000000000000000 P a7af291ee1c443908e5c7a88a871b896 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 02:47:24.995095 3572 raft_consensus.cc:482] T 00000000000000000000000000000000 P a7af291ee1c443908e5c7a88a871b896 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 02:47:24.995164 3572 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a7af291ee1c443908e5c7a88a871b896 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:47:24.995990 3572 raft_consensus.cc:504] T 00000000000000000000000000000000 P a7af291ee1c443908e5c7a88a871b896 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7af291ee1c443908e5c7a88a871b896" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 36527 } }
I20260925 02:47:24.996274 3572 leader_election.cc:302] T 00000000000000000000000000000000 P a7af291ee1c443908e5c7a88a871b896 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a7af291ee1c443908e5c7a88a871b896; no voters:
I20260925 02:47:24.996531 3572 leader_election.cc:288] T 00000000000000000000000000000000 P a7af291ee1c443908e5c7a88a871b896 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 02:47:24.996662 3582 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a7af291ee1c443908e5c7a88a871b896 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:47:24.996906 3582 raft_consensus.cc:686] T 00000000000000000000000000000000 P a7af291ee1c443908e5c7a88a871b896 [term 1 LEADER]: Becoming Leader. State: Replica: a7af291ee1c443908e5c7a88a871b896, State: Running, Role: LEADER
I20260925 02:47:24.997200 3582 consensus_queue.cc:237] T 00000000000000000000000000000000 P a7af291ee1c443908e5c7a88a871b896 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7af291ee1c443908e5c7a88a871b896" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 36527 } }
I20260925 02:47:24.997623 3572 sys_catalog.cc:564] T 00000000000000000000000000000000 P a7af291ee1c443908e5c7a88a871b896 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:47:24.998855 3584 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7af291ee1c443908e5c7a88a871b896 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a7af291ee1c443908e5c7a88a871b896. Latest consensus state: current_term: 1 leader_uuid: "a7af291ee1c443908e5c7a88a871b896" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7af291ee1c443908e5c7a88a871b896" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 36527 } } }
I20260925 02:47:24.999002 3584 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7af291ee1c443908e5c7a88a871b896 [sys.catalog]: This master's current role is: LEADER
I20260925 02:47:24.999382 3583 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7af291ee1c443908e5c7a88a871b896 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a7af291ee1c443908e5c7a88a871b896" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7af291ee1c443908e5c7a88a871b896" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 36527 } } }
I20260925 02:47:24.999467 3583 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7af291ee1c443908e5c7a88a871b896 [sys.catalog]: This master's current role is: LEADER
W20260925 02:47:25.002018 3597 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a7af291ee1c443908e5c7a88a871b896: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 02:47:25.002089 3597 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 02:47:25.002162 3593 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:47:25.002960 3593 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:47:25.008143 3593 catalog_manager.cc:1347] Generated new cluster ID: 38c5e5200e2a4be4917c9177b3f67646
I20260925 02:47:25.008204 3593 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 02:47:25.031641 3593 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 02:47:25.031985 3593 catalog_manager.cc:1504] Loading token signing keys...
I20260925 02:47:25.040619 3593 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a7af291ee1c443908e5c7a88a871b896: Generated new TSK 0
I20260925 02:47:25.041313 3593 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 02:47:25.045022 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 3504
2026-09-25T02:47:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1304 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790304178325258-13942-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 02:47:25 dist-test-slave-kznw krb5kdc[3603](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 02:47:25 dist-test-slave-kznw krb5kdc[3603](info): set up 2 sockets
Sep 25 02:47:25 dist-test-slave-kznw krb5kdc[3603](info): commencing operation
krb5kdc: starting...
W20260925 02:47:26.002514 13942 mini_kdc.cc:121] Time spent starting KDC: real 0.938s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790304178325258-13942-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790304178325258-13942-0/krb5kdc/test-user.keytab.
Sep 25 02:47:26 dist-test-slave-kznw krb5kdc[3603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304446, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T02:47:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T02:47:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.157.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.157.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.157.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab.
Entry for principal kudu/127.13.157.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab.
I20260925 02:47:26.126070 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790304178325258-13942-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790304178325258-13942-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790304178325258-13942-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.157.190:46279
--webserver_interface=127.13.157.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab
--principal=kudu/127.13.157.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:33981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.157.190:46279
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:47:26.236531 3616 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:47:26.236806 3616 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:47:26.236847 3616 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:47:26.240869 3616 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:47:26.240931 3616 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:47:26.240957 3616 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:47:26.240972 3616 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:47:26.240986 3616 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:47:26.246313 3616 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:33981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790304178325258-13942-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790304178325258-13942-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.157.190:46279
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab
--principal=kudu/127.13.157.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.157.190:46279
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.157.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3616
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790304178325258-13942-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:47:26.247506 3616 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:47:26.248555 3616 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:47:26.257166 3621 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:47:26.257248 3616 server_base.cc:1034] running on GCE node
W20260925 02:47:26.257186 3624 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:47:26.257166 3622 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:47:26.257987 3616 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:47:26.259209 3616 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:47:26.260385 3616 hybrid_clock.cc:648] HybridClock initialized: now 1790304446260360 us; error 38 us; skew 500 ppm
Sep 25 02:47:26 dist-test-slave-kznw krb5kdc[3603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304446, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:47:26.263120 3616 init.cc:377] Logged in from keytab as kudu/127.13.157.190@KRBTEST.COM (short username kudu)
I20260925 02:47:26.264038 3616 webserver.cc:460] Webserver started at http://127.13.157.190:37179/ using document root <none> and password file <none>
I20260925 02:47:26.264576 3616 fs_manager.cc:359] Metadata directory not provided
I20260925 02:47:26.264642 3616 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:47:26.264845 3616 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:47:26.266880 3616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790304178325258-13942-0/minicluster-data/master-0/data/instance:
uuid: "2ffb6c0faa204b39952203578a7503a2"
format_stamp: "Formatted at 2026-09-25 02:47:26 on dist-test-slave-kznw"
server_key: "01ccce379a652b67dd93179db7731616"
server_key_iv: "9342b0bd4aeb613516258a36c706d8d9"
server_key_version: "encryptionkey@0"
I20260925 02:47:26.267481 3616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790304178325258-13942-0/minicluster-data/master-0/wal/instance:
uuid: "2ffb6c0faa204b39952203578a7503a2"
format_stamp: "Formatted at 2026-09-25 02:47:26 on dist-test-slave-kznw"
server_key: "01ccce379a652b67dd93179db7731616"
server_key_iv: "9342b0bd4aeb613516258a36c706d8d9"
server_key_version: "encryptionkey@0"
I20260925 02:47:26.271253 3616 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260925 02:47:26.273958 3631 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:47:26.275347 3616 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 02:47:26.275504 3616 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790304178325258-13942-0/minicluster-data/master-0/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790304178325258-13942-0/minicluster-data/master-0/wal
uuid: "2ffb6c0faa204b39952203578a7503a2"
format_stamp: "Formatted at 2026-09-25 02:47:26 on dist-test-slave-kznw"
server_key: "01ccce379a652b67dd93179db7731616"
server_key_iv: "9342b0bd4aeb613516258a36c706d8d9"
server_key_version: "encryptionkey@0"
I20260925 02:47:26.275609 3616 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790304178325258-13942-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790304178325258-13942-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790304178325258-13942-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:47:26.289645 3616 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:47:26.292877 3616 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:47:26.293068 3616 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:47:26.300149 3616 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.190:46279
I20260925 02:47:26.300200 3683 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.190:46279 every 8 connection(s)
I20260925 02:47:26.301100 3616 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
I20260925 02:47:26.302714 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 3616
I20260925 02:47:26.302879 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790304178325258-13942-0/minicluster-data/master-0/wal/instance
I20260925 02:47:26.303263 13942 external_mini_cluster.cc:1428] Setting key 2be6e41db04f014df7b93db79d593c3c
I20260925 02:47:26.304390 3684 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:47:26.310351 3684 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ffb6c0faa204b39952203578a7503a2: Bootstrap starting.
Sep 25 02:47:26 dist-test-slave-kznw krb5kdc[3603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304446, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.157.190@KRBTEST.COM
I20260925 02:47:26.312968 3684 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2ffb6c0faa204b39952203578a7503a2: Neither blocks nor log segments found. Creating new log.
I20260925 02:47:26.313728 3684 log.cc:824] T 00000000000000000000000000000000 P 2ffb6c0faa204b39952203578a7503a2: Log is configured to *not* fsync() on all Append() calls
I20260925 02:47:26.316084 3684 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ffb6c0faa204b39952203578a7503a2: No bootstrap required, opened a new log
I20260925 02:47:26.317595 3687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:26.304684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53065 (local address 127.13.157.190:46279)
0925 02:47:26.305172 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:26.305187 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:26.305224 (+ 37us) server_negotiation.cc:407] Connection header received
0925 02:47:26.305872 (+ 648us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:26.305889 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:26.306242 (+ 353us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:26.306553 (+ 311us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:47:26.307370 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:26.308375 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:26.309234 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:26.309875 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:26.310055 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:26.312837 (+ 2782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:47:26.312842 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:47:26.312856 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:47:26.312884 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:47:26.314671 (+ 1787us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:47:26.315196 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:47:26.315201 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:47:26.315205 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:47:26.315320 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:47:26.315679 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:47:26.315681 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:47:26.315682 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:47:26.316083 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:47:26.316257 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:26.316567 (+ 310us) server_negotiation.cc:299] Negotiation successful
0925 02:47:26.316855 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":156,"threads_started":1}
I20260925 02:47:26.320122 3684 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2ffb6c0faa204b39952203578a7503a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ffb6c0faa204b39952203578a7503a2" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 46279 } }
I20260925 02:47:26.320417 3684 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2ffb6c0faa204b39952203578a7503a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:47:26.320487 3684 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2ffb6c0faa204b39952203578a7503a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ffb6c0faa204b39952203578a7503a2, State: Initialized, Role: FOLLOWER
I20260925 02:47:26.320727 13942 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 02:47:26.320963 3684 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2ffb6c0faa204b39952203578a7503a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ffb6c0faa204b39952203578a7503a2" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 46279 } }
I20260925 02:47:26.321120 3684 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2ffb6c0faa204b39952203578a7503a2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 02:47:26.321180 3684 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2ffb6c0faa204b39952203578a7503a2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 02:47:26.321281 3684 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2ffb6c0faa204b39952203578a7503a2 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:47:26.322459 3684 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2ffb6c0faa204b39952203578a7503a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ffb6c0faa204b39952203578a7503a2" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 46279 } }
I20260925 02:47:26.322816 3684 leader_election.cc:302] T 00000000000000000000000000000000 P 2ffb6c0faa204b39952203578a7503a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2ffb6c0faa204b39952203578a7503a2; no voters:
I20260925 02:47:26.323155 3684 leader_election.cc:288] T 00000000000000000000000000000000 P 2ffb6c0faa204b39952203578a7503a2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 02:47:26.323529 3695 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2ffb6c0faa204b39952203578a7503a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:47:26.323787 3695 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2ffb6c0faa204b39952203578a7503a2 [term 1 LEADER]: Becoming Leader. State: Replica: 2ffb6c0faa204b39952203578a7503a2, State: Running, Role: LEADER
I20260925 02:47:26.324103 3695 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2ffb6c0faa204b39952203578a7503a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ffb6c0faa204b39952203578a7503a2" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 46279 } }
I20260925 02:47:26.324566 3684 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2ffb6c0faa204b39952203578a7503a2 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:47:26.325747 3694 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ffb6c0faa204b39952203578a7503a2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2ffb6c0faa204b39952203578a7503a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ffb6c0faa204b39952203578a7503a2" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 46279 } } }
I20260925 02:47:26.325872 3694 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ffb6c0faa204b39952203578a7503a2 [sys.catalog]: This master's current role is: LEADER
I20260925 02:47:26.326153 3696 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ffb6c0faa204b39952203578a7503a2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2ffb6c0faa204b39952203578a7503a2. Latest consensus state: current_term: 1 leader_uuid: "2ffb6c0faa204b39952203578a7503a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ffb6c0faa204b39952203578a7503a2" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 46279 } } }
I20260925 02:47:26.326294 3696 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ffb6c0faa204b39952203578a7503a2 [sys.catalog]: This master's current role is: LEADER
I20260925 02:47:26.326592 3703 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:47:26.330269 3703 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:47:26.332371 3687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:26.323004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38569 (local address 127.13.157.190:46279)
0925 02:47:26.323153 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:26.323157 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:26.323310 (+ 153us) server_negotiation.cc:407] Connection header received
0925 02:47:26.323453 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:26.323456 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:26.323514 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:26.323601 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:47:26.324544 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:26.325325 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:26.326447 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:26.326785 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:26.326885 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:26.328580 (+ 1695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:47:26.328584 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:47:26.328587 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:47:26.328612 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:47:26.330142 (+ 1530us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:47:26.330627 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:47:26.330631 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:47:26.330632 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:47:26.330720 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:47:26.331046 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:47:26.331049 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:47:26.331051 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:47:26.331260 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:47:26.331374 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:26.332094 (+ 720us) server_negotiation.cc:299] Negotiation successful
0925 02:47:26.332210 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260925 02:47:26.336673 3703 catalog_manager.cc:1347] Generated new cluster ID: 3927b1d6a49c4dc9aaa9ec1588c644aa
I20260925 02:47:26.336803 3703 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 02:47:26.351432 3703 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 02:47:26.352394 3703 catalog_manager.cc:1504] Loading token signing keys...
I20260925 02:47:26.362002 3703 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2ffb6c0faa204b39952203578a7503a2: Generated new TSK 0
I20260925 02:47:26.362785 3703 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 02:47:26.375720 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 3616
2026-09-25T02:47:26Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1330 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790304178325258-13942-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 02:47:26 dist-test-slave-kznw krb5kdc[3715](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 02:47:26 dist-test-slave-kznw krb5kdc[3715](info): set up 2 sockets
Sep 25 02:47:26 dist-test-slave-kznw krb5kdc[3715](info): commencing operation
krb5kdc: starting...
W20260925 02:47:27.330646 13942 mini_kdc.cc:121] Time spent starting KDC: real 0.936s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790304178325258-13942-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790304178325258-13942-0/krb5kdc/test-user.keytab.
Sep 25 02:47:27 dist-test-slave-kznw krb5kdc[3715](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304447, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T02:47:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T02:47:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.157.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.157.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.157.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab.
Entry for principal kudu/127.13.157.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab.
I20260925 02:47:27.451233 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790304178325258-13942-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790304178325258-13942-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790304178325258-13942-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.157.190:33451
--webserver_interface=127.13.157.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab
--principal=kudu/127.13.157.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:44771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.157.190:33451
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:47:27.558845 3728 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:47:27.559147 3728 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:47:27.559206 3728 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:47:27.563162 3728 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:47:27.563220 3728 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:47:27.563246 3728 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:47:27.563268 3728 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:47:27.563297 3728 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:47:27.568432 3728 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:44771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790304178325258-13942-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790304178325258-13942-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.157.190:33451
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab
--principal=kudu/127.13.157.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.157.190:33451
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.157.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3728
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790304178325258-13942-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:47:27.569434 3728 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:47:27.570427 3728 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:47:27.578883 3734 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:47:27.578856 3736 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:47:27.578873 3733 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:47:27.579337 3728 server_base.cc:1034] running on GCE node
I20260925 02:47:27.579867 3728 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:47:27.580945 3728 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:47:27.582100 3728 hybrid_clock.cc:648] HybridClock initialized: now 1790304447582075 us; error 37 us; skew 500 ppm
Sep 25 02:47:27 dist-test-slave-kznw krb5kdc[3715](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304447, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:47:27.589071 3728 init.cc:377] Logged in from keytab as kudu/127.13.157.190@KRBTEST.COM (short username kudu)
I20260925 02:47:27.589987 3728 webserver.cc:460] Webserver started at http://127.13.157.190:33407/ using document root <none> and password file <none>
I20260925 02:47:27.590523 3728 fs_manager.cc:359] Metadata directory not provided
I20260925 02:47:27.590591 3728 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:47:27.590792 3728 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:47:27.592849 3728 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790304178325258-13942-0/minicluster-data/master-0/data/instance:
uuid: "15ec8bd6048941908c9137de7ff9dea1"
format_stamp: "Formatted at 2026-09-25 02:47:27 on dist-test-slave-kznw"
server_key: "7998bce11f639a95068971faee9a2029"
server_key_iv: "079aed4f9bd5a00c0b8b504f812d9adc"
server_key_version: "encryptionkey@0"
I20260925 02:47:27.593400 3728 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790304178325258-13942-0/minicluster-data/master-0/wal/instance:
uuid: "15ec8bd6048941908c9137de7ff9dea1"
format_stamp: "Formatted at 2026-09-25 02:47:27 on dist-test-slave-kznw"
server_key: "7998bce11f639a95068971faee9a2029"
server_key_iv: "079aed4f9bd5a00c0b8b504f812d9adc"
server_key_version: "encryptionkey@0"
I20260925 02:47:27.597093 3728 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260925 02:47:27.599506 3743 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:47:27.600769 3728 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260925 02:47:27.600883 3728 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790304178325258-13942-0/minicluster-data/master-0/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790304178325258-13942-0/minicluster-data/master-0/wal
uuid: "15ec8bd6048941908c9137de7ff9dea1"
format_stamp: "Formatted at 2026-09-25 02:47:27 on dist-test-slave-kznw"
server_key: "7998bce11f639a95068971faee9a2029"
server_key_iv: "079aed4f9bd5a00c0b8b504f812d9adc"
server_key_version: "encryptionkey@0"
I20260925 02:47:27.600978 3728 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790304178325258-13942-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790304178325258-13942-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790304178325258-13942-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:47:27.621193 3728 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:47:27.624255 3728 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:47:27.624439 3728 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:47:27.631295 3728 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.190:33451
I20260925 02:47:27.631359 3795 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.190:33451 every 8 connection(s)
I20260925 02:47:27.632283 3728 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
I20260925 02:47:27.635066 3796 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:47:27.637390 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 3728
I20260925 02:47:27.637506 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790304178325258-13942-0/minicluster-data/master-0/wal/instance
I20260925 02:47:27.637768 13942 external_mini_cluster.cc:1428] Setting key 53b296cb3549b0bf2ca35bd0c4b00a03
I20260925 02:47:27.641031 3796 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15ec8bd6048941908c9137de7ff9dea1: Bootstrap starting.
I20260925 02:47:27.643301 3796 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 15ec8bd6048941908c9137de7ff9dea1: Neither blocks nor log segments found. Creating new log.
I20260925 02:47:27.644029 3796 log.cc:824] T 00000000000000000000000000000000 P 15ec8bd6048941908c9137de7ff9dea1: Log is configured to *not* fsync() on all Append() calls
I20260925 02:47:27.646601 3799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:27.639229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43609 (local address 127.13.157.190:33451)
0925 02:47:27.639720 (+ 491us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:27.639735 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:27.639770 (+ 35us) server_negotiation.cc:407] Connection header received
0925 02:47:27.640398 (+ 628us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:27.640414 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:27.640763 (+ 349us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:27.641068 (+ 305us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:47:27.641466 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:27.642444 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:27.643357 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:27.643854 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:27.644009 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:27.644834 (+ 825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:47:27.644839 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:47:27.644852 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:47:27.645247 (+ 395us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:47:27.645302 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:47:27.645324 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:47:27.645410 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:27.645628 (+ 218us) server_negotiation.cc:299] Negotiation successful
0925 02:47:27.645828 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":347,"thread_start_us":137,"threads_started":1}
I20260925 02:47:27.646796 3796 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15ec8bd6048941908c9137de7ff9dea1: No bootstrap required, opened a new log
I20260925 02:47:27.649641 13942 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 02:47:27.650242 3796 raft_consensus.cc:348] T 00000000000000000000000000000000 P 15ec8bd6048941908c9137de7ff9dea1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15ec8bd6048941908c9137de7ff9dea1" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 33451 } }
I20260925 02:47:27.650539 3796 raft_consensus.cc:374] T 00000000000000000000000000000000 P 15ec8bd6048941908c9137de7ff9dea1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:47:27.650628 3796 raft_consensus.cc:729] T 00000000000000000000000000000000 P 15ec8bd6048941908c9137de7ff9dea1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15ec8bd6048941908c9137de7ff9dea1, State: Initialized, Role: FOLLOWER
I20260925 02:47:27.651116 3796 consensus_queue.cc:260] T 00000000000000000000000000000000 P 15ec8bd6048941908c9137de7ff9dea1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15ec8bd6048941908c9137de7ff9dea1" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 33451 } }
I20260925 02:47:27.651273 3796 raft_consensus.cc:388] T 00000000000000000000000000000000 P 15ec8bd6048941908c9137de7ff9dea1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 02:47:27.651377 3796 raft_consensus.cc:482] T 00000000000000000000000000000000 P 15ec8bd6048941908c9137de7ff9dea1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 02:47:27.651507 3796 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 15ec8bd6048941908c9137de7ff9dea1 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:47:27.652663 3796 raft_consensus.cc:504] T 00000000000000000000000000000000 P 15ec8bd6048941908c9137de7ff9dea1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15ec8bd6048941908c9137de7ff9dea1" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 33451 } }
I20260925 02:47:27.653048 3796 leader_election.cc:302] T 00000000000000000000000000000000 P 15ec8bd6048941908c9137de7ff9dea1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 15ec8bd6048941908c9137de7ff9dea1; no voters:
I20260925 02:47:27.653360 3796 leader_election.cc:288] T 00000000000000000000000000000000 P 15ec8bd6048941908c9137de7ff9dea1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 02:47:27.653468 3806 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 15ec8bd6048941908c9137de7ff9dea1 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:47:27.653658 3806 raft_consensus.cc:686] T 00000000000000000000000000000000 P 15ec8bd6048941908c9137de7ff9dea1 [term 1 LEADER]: Becoming Leader. State: Replica: 15ec8bd6048941908c9137de7ff9dea1, State: Running, Role: LEADER
I20260925 02:47:27.654011 3806 consensus_queue.cc:237] T 00000000000000000000000000000000 P 15ec8bd6048941908c9137de7ff9dea1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15ec8bd6048941908c9137de7ff9dea1" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 33451 } }
I20260925 02:47:27.654732 3796 sys_catalog.cc:564] T 00000000000000000000000000000000 P 15ec8bd6048941908c9137de7ff9dea1 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:47:27.655963 3807 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15ec8bd6048941908c9137de7ff9dea1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "15ec8bd6048941908c9137de7ff9dea1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15ec8bd6048941908c9137de7ff9dea1" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 33451 } } }
I20260925 02:47:27.656085 3807 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15ec8bd6048941908c9137de7ff9dea1 [sys.catalog]: This master's current role is: LEADER
I20260925 02:47:27.656301 3807 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15ec8bd6048941908c9137de7ff9dea1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 15ec8bd6048941908c9137de7ff9dea1. Latest consensus state: current_term: 1 leader_uuid: "15ec8bd6048941908c9137de7ff9dea1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15ec8bd6048941908c9137de7ff9dea1" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 33451 } } }
I20260925 02:47:27.656378 3807 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15ec8bd6048941908c9137de7ff9dea1 [sys.catalog]: This master's current role is: LEADER
I20260925 02:47:27.656708 3799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:27.652169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55453 (local address 127.13.157.190:33451)
0925 02:47:27.652271 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:27.652276 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:27.652524 (+ 248us) server_negotiation.cc:407] Connection header received
0925 02:47:27.652636 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:27.652639 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:27.652720 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:27.652787 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:47:27.653167 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:27.653897 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:27.654762 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:27.655124 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:27.655215 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:27.656141 (+ 926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:47:27.656144 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:47:27.656147 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:47:27.656159 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:47:27.656188 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:47:27.656201 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:47:27.656256 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:27.656552 (+ 296us) server_negotiation.cc:299] Negotiation successful
0925 02:47:27.656608 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
W20260925 02:47:27.659596 3822 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 15ec8bd6048941908c9137de7ff9dea1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 02:47:27.659659 3822 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 02:47:27.659740 3815 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:47:27.660658 3815 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:47:27.666119 3815 catalog_manager.cc:1347] Generated new cluster ID: e307d0081cf542209c52396ad7ae8716
I20260925 02:47:27.666185 3815 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 02:47:27.677399 3815 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 02:47:27.678416 3815 catalog_manager.cc:1504] Loading token signing keys...
I20260925 02:47:27.681835 3815 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 15ec8bd6048941908c9137de7ff9dea1: Generated new TSK 0
I20260925 02:47:27.682480 3815 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 02:47:27.686699 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 3728
2026-09-25T02:47:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1310 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790304178325258-13942-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 02:47:27 dist-test-slave-kznw krb5kdc[3827](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 02:47:27 dist-test-slave-kznw krb5kdc[3827](info): set up 2 sockets
Sep 25 02:47:27 dist-test-slave-kznw krb5kdc[3827](info): commencing operation
krb5kdc: starting...
W20260925 02:47:28.638494 13942 mini_kdc.cc:121] Time spent starting KDC: real 0.933s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790304178325258-13942-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790304178325258-13942-0/krb5kdc/test-user.keytab.
Sep 25 02:47:28 dist-test-slave-kznw krb5kdc[3827](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304448, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T02:47:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T02:47:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.157.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.157.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.157.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab.
Entry for principal kudu/127.13.157.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab.
I20260925 02:47:28.752041 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790304178325258-13942-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790304178325258-13942-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790304178325258-13942-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.157.190:36467
--webserver_interface=127.13.157.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab
--principal=kudu/127.13.157.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:32931
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.157.190:36467
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:47:28.859105 3840 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:47:28.859362 3840 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:47:28.859419 3840 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:47:28.863358 3840 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:47:28.863415 3840 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:47:28.863440 3840 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:47:28.863462 3840 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:47:28.863492 3840 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:47:28.868583 3840 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:32931
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790304178325258-13942-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790304178325258-13942-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.157.190:36467
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab
--principal=kudu/127.13.157.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.157.190:36467
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.157.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3840
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790304178325258-13942-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:47:28.869599 3840 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:47:28.870560 3840 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:47:28.878747 3848 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:47:28.878820 3846 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:47:28.878750 3845 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:47:28.879245 3840 server_base.cc:1034] running on GCE node
I20260925 02:47:28.879769 3840 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:47:28.880836 3840 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:47:28.881994 3840 hybrid_clock.cc:648] HybridClock initialized: now 1790304448881978 us; error 28 us; skew 500 ppm
Sep 25 02:47:28 dist-test-slave-kznw krb5kdc[3827](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304448, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:47:28.884586 3840 init.cc:377] Logged in from keytab as kudu/127.13.157.190@KRBTEST.COM (short username kudu)
I20260925 02:47:28.885460 3840 webserver.cc:460] Webserver started at http://127.13.157.190:45789/ using document root <none> and password file <none>
I20260925 02:47:28.885982 3840 fs_manager.cc:359] Metadata directory not provided
I20260925 02:47:28.886055 3840 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:47:28.886248 3840 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:47:28.888307 3840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790304178325258-13942-0/minicluster-data/master-0/data/instance:
uuid: "31619008e27248f5a09ad9c52e0c34f3"
format_stamp: "Formatted at 2026-09-25 02:47:28 on dist-test-slave-kznw"
server_key: "5b17f5f8b7aafe8fee189a97f92b8e7b"
server_key_iv: "68e4ad114503a544501a7ea2ec991c45"
server_key_version: "encryptionkey@0"
I20260925 02:47:28.888907 3840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790304178325258-13942-0/minicluster-data/master-0/wal/instance:
uuid: "31619008e27248f5a09ad9c52e0c34f3"
format_stamp: "Formatted at 2026-09-25 02:47:28 on dist-test-slave-kznw"
server_key: "5b17f5f8b7aafe8fee189a97f92b8e7b"
server_key_iv: "68e4ad114503a544501a7ea2ec991c45"
server_key_version: "encryptionkey@0"
I20260925 02:47:28.892575 3840 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260925 02:47:28.894941 3855 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:47:28.896131 3840 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260925 02:47:28.896241 3840 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790304178325258-13942-0/minicluster-data/master-0/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790304178325258-13942-0/minicluster-data/master-0/wal
uuid: "31619008e27248f5a09ad9c52e0c34f3"
format_stamp: "Formatted at 2026-09-25 02:47:28 on dist-test-slave-kznw"
server_key: "5b17f5f8b7aafe8fee189a97f92b8e7b"
server_key_iv: "68e4ad114503a544501a7ea2ec991c45"
server_key_version: "encryptionkey@0"
I20260925 02:47:28.896333 3840 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790304178325258-13942-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790304178325258-13942-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790304178325258-13942-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:47:28.900734 3840 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:47:28.903510 3840 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:47:28.903668 3840 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:47:28.910279 3840 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.190:36467
I20260925 02:47:28.910341 3907 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.190:36467 every 8 connection(s)
I20260925 02:47:28.911242 3840 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
I20260925 02:47:28.913964 3908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:47:28.917466 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 3840
I20260925 02:47:28.917609 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790304178325258-13942-0/minicluster-data/master-0/wal/instance
I20260925 02:47:28.917897 13942 external_mini_cluster.cc:1428] Setting key 713ddfd29d80d4a5c432b0bdd301a451
I20260925 02:47:28.920056 3908 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31619008e27248f5a09ad9c52e0c34f3: Bootstrap starting.
I20260925 02:47:28.922287 3908 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 31619008e27248f5a09ad9c52e0c34f3: Neither blocks nor log segments found. Creating new log.
I20260925 02:47:28.923032 3908 log.cc:824] T 00000000000000000000000000000000 P 31619008e27248f5a09ad9c52e0c34f3: Log is configured to *not* fsync() on all Append() calls
I20260925 02:47:28.923233 3911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:28.919368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45797 (local address 127.13.157.190:36467)
0925 02:47:28.919825 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:28.919840 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:28.919890 (+ 50us) server_negotiation.cc:407] Connection header received
0925 02:47:28.920544 (+ 654us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:28.920564 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:28.920860 (+ 296us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:28.921205 (+ 345us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:47:28.922007 (+ 802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:47:28.922012 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:47:28.922023 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:47:28.922034 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:47:28.922085 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:47:28.922105 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:47:28.922194 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:28.922300 (+ 106us) server_negotiation.cc:299] Negotiation successful
0925 02:47:28.922459 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":137,"threads_started":1}
I20260925 02:47:28.925645 3908 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31619008e27248f5a09ad9c52e0c34f3: No bootstrap required, opened a new log
I20260925 02:47:28.925796 13942 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 02:47:28.929442 3911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:28.928004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54017 (local address 127.13.157.190:36467)
0925 02:47:28.928120 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:28.928124 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:28.928265 (+ 141us) server_negotiation.cc:407] Connection header received
0925 02:47:28.928350 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:28.928353 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:28.928388 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:28.928467 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:47:28.929113 (+ 646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:47:28.929116 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:47:28.929118 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:47:28.929131 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:47:28.929158 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:47:28.929172 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:47:28.929260 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:28.929291 (+ 31us) server_negotiation.cc:299] Negotiation successful
0925 02:47:28.929325 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260925 02:47:28.930008 3908 raft_consensus.cc:348] T 00000000000000000000000000000000 P 31619008e27248f5a09ad9c52e0c34f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31619008e27248f5a09ad9c52e0c34f3" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 36467 } }
I20260925 02:47:28.930238 3908 raft_consensus.cc:374] T 00000000000000000000000000000000 P 31619008e27248f5a09ad9c52e0c34f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:47:28.930317 3908 raft_consensus.cc:729] T 00000000000000000000000000000000 P 31619008e27248f5a09ad9c52e0c34f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31619008e27248f5a09ad9c52e0c34f3, State: Initialized, Role: FOLLOWER
I20260925 02:47:28.930737 3908 consensus_queue.cc:260] T 00000000000000000000000000000000 P 31619008e27248f5a09ad9c52e0c34f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31619008e27248f5a09ad9c52e0c34f3" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 36467 } }
I20260925 02:47:28.930864 3908 raft_consensus.cc:388] T 00000000000000000000000000000000 P 31619008e27248f5a09ad9c52e0c34f3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 02:47:28.930945 3908 raft_consensus.cc:482] T 00000000000000000000000000000000 P 31619008e27248f5a09ad9c52e0c34f3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 02:47:28.931043 3908 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 31619008e27248f5a09ad9c52e0c34f3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:47:28.931962 3908 raft_consensus.cc:504] T 00000000000000000000000000000000 P 31619008e27248f5a09ad9c52e0c34f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31619008e27248f5a09ad9c52e0c34f3" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 36467 } }
I20260925 02:47:28.932297 3908 leader_election.cc:302] T 00000000000000000000000000000000 P 31619008e27248f5a09ad9c52e0c34f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 31619008e27248f5a09ad9c52e0c34f3; no voters:
I20260925 02:47:28.932597 3908 leader_election.cc:288] T 00000000000000000000000000000000 P 31619008e27248f5a09ad9c52e0c34f3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 02:47:28.932695 3918 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 31619008e27248f5a09ad9c52e0c34f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:47:28.932899 3918 raft_consensus.cc:686] T 00000000000000000000000000000000 P 31619008e27248f5a09ad9c52e0c34f3 [term 1 LEADER]: Becoming Leader. State: Replica: 31619008e27248f5a09ad9c52e0c34f3, State: Running, Role: LEADER
I20260925 02:47:28.933240 3918 consensus_queue.cc:237] T 00000000000000000000000000000000 P 31619008e27248f5a09ad9c52e0c34f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31619008e27248f5a09ad9c52e0c34f3" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 36467 } }
I20260925 02:47:28.933779 3908 sys_catalog.cc:564] T 00000000000000000000000000000000 P 31619008e27248f5a09ad9c52e0c34f3 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:47:28.934792 3919 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31619008e27248f5a09ad9c52e0c34f3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "31619008e27248f5a09ad9c52e0c34f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31619008e27248f5a09ad9c52e0c34f3" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 36467 } } }
I20260925 02:47:28.934931 3919 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31619008e27248f5a09ad9c52e0c34f3 [sys.catalog]: This master's current role is: LEADER
I20260925 02:47:28.935259 3920 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31619008e27248f5a09ad9c52e0c34f3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 31619008e27248f5a09ad9c52e0c34f3. Latest consensus state: current_term: 1 leader_uuid: "31619008e27248f5a09ad9c52e0c34f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31619008e27248f5a09ad9c52e0c34f3" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 36467 } } }
I20260925 02:47:28.935326 3927 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:47:28.935407 3920 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31619008e27248f5a09ad9c52e0c34f3 [sys.catalog]: This master's current role is: LEADER
I20260925 02:47:28.938354 3927 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:47:28.943476 3927 catalog_manager.cc:1347] Generated new cluster ID: 02d3c9889e6647f29ee1d61f7acb9b70
I20260925 02:47:28.943552 3927 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 02:47:28.969870 3927 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 02:47:28.970206 3927 catalog_manager.cc:1504] Loading token signing keys...
I20260925 02:47:28.974195 3927 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 31619008e27248f5a09ad9c52e0c34f3: Generated new TSK 0
I20260925 02:47:28.974817 3927 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 02:47:28.983978 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 3840
2026-09-25T02:47:28Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1298 ms)
[----------] 18 tests from AuthTokenIssuingTest (23752 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790304178325258-13942-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 02:47:29 dist-test-slave-kznw krb5kdc[3939](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 02:47:29 dist-test-slave-kznw krb5kdc[3939](info): set up 2 sockets
Sep 25 02:47:29 dist-test-slave-kznw krb5kdc[3939](info): commencing operation
krb5kdc: starting...
W20260925 02:47:29.946959 13942 mini_kdc.cc:121] Time spent starting KDC: real 0.943s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790304178325258-13942-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790304178325258-13942-0/krb5kdc/test-user.keytab.
Sep 25 02:47:30 dist-test-slave-kznw krb5kdc[3939](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304450, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T02:47:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T02:47:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.157.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.157.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.157.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab.
Entry for principal kudu/127.13.157.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab.
I20260925 02:47:30.069764 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790304178325258-13942-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790304178325258-13942-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790304178325258-13942-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.157.190:43651
--webserver_interface=127.13.157.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab
--principal=kudu/127.13.157.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:43127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.157.190:43651,127.13.157.189:42211,127.13.157.188:33993
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:47:30.183302 3952 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260925 02:47:30.183599 3952 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:47:30.183622 3952 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:47:30.183672 3952 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:47:30.187640 3952 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:47:30.187702 3952 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:47:30.187721 3952 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:47:30.187736 3952 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:47:30.187748 3952 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:47:30.193312 3952 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:43127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790304178325258-13942-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790304178325258-13942-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.157.190:43651,127.13.157.189:42211,127.13.157.188:33993
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab
--principal=kudu/127.13.157.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.157.190:43651
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.157.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3952
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790304178325258-13942-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:47:30.194515 3952 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:47:30.195590 3952 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:47:30.203984 3957 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:47:30.204006 3958 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:47:30.204046 3952 server_base.cc:1034] running on GCE node
W20260925 02:47:30.204056 3960 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:47:30.204749 3952 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:47:30.205861 3952 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:47:30.206979 3952 hybrid_clock.cc:648] HybridClock initialized: now 1790304450206953 us; error 25 us; skew 500 ppm
Sep 25 02:47:30 dist-test-slave-kznw krb5kdc[3939](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304450, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:47:30.209525 3952 init.cc:377] Logged in from keytab as kudu/127.13.157.190@KRBTEST.COM (short username kudu)
I20260925 02:47:30.210371 3952 webserver.cc:460] Webserver started at http://127.13.157.190:44475/ using document root <none> and password file <none>
I20260925 02:47:30.210880 3952 fs_manager.cc:359] Metadata directory not provided
I20260925 02:47:30.210991 3952 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:47:30.211199 3952 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:47:30.213195 3952 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790304178325258-13942-0/minicluster-data/master-0/data/instance:
uuid: "55a954a30ce64942959f12b90c70cae4"
format_stamp: "Formatted at 2026-09-25 02:47:30 on dist-test-slave-kznw"
server_key: "dda667b562f2d0fee8b39e9623444a11"
server_key_iv: "513f3876fb0dcb90641c17ef2f4d4b96"
server_key_version: "encryptionkey@0"
I20260925 02:47:30.213729 3952 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790304178325258-13942-0/minicluster-data/master-0/wal/instance:
uuid: "55a954a30ce64942959f12b90c70cae4"
format_stamp: "Formatted at 2026-09-25 02:47:30 on dist-test-slave-kznw"
server_key: "dda667b562f2d0fee8b39e9623444a11"
server_key_iv: "513f3876fb0dcb90641c17ef2f4d4b96"
server_key_version: "encryptionkey@0"
I20260925 02:47:30.217345 3952 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260925 02:47:30.219630 3967 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:47:30.220801 3952 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260925 02:47:30.220913 3952 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790304178325258-13942-0/minicluster-data/master-0/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790304178325258-13942-0/minicluster-data/master-0/wal
uuid: "55a954a30ce64942959f12b90c70cae4"
format_stamp: "Formatted at 2026-09-25 02:47:30 on dist-test-slave-kznw"
server_key: "dda667b562f2d0fee8b39e9623444a11"
server_key_iv: "513f3876fb0dcb90641c17ef2f4d4b96"
server_key_version: "encryptionkey@0"
I20260925 02:47:30.221020 3952 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790304178325258-13942-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790304178325258-13942-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790304178325258-13942-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:47:30.243888 3952 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:47:30.247043 3952 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:47:30.247238 3952 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:47:30.255139 3952 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.190:43651
I20260925 02:47:30.255447 4019 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.190:43651 every 8 connection(s)
I20260925 02:47:30.256224 3952 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
I20260925 02:47:30.259012 4020 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:47:30.262707 4020 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.157.190" port: 43651 } has no permanent_uuid. Determining permanent_uuid...
I20260925 02:47:30.266422 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 3952
I20260925 02:47:30.266564 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790304178325258-13942-0/minicluster-data/master-0/wal/instance
I20260925 02:47:30.266822 13942 external_mini_cluster.cc:1428] Setting key f78c4d9f48d8fad4c299b4bc096e603b
Sep 25 02:47:30 dist-test-slave-kznw krb5kdc[3939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304450, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.190@KRBTEST.COM for kudu/127.13.157.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.157.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.157.189@KRBTEST.COM" created.
I20260925 02:47:30.278352 4022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:30.264976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51908 (local address 127.13.157.190:43651)
0925 02:47:30.265504 (+ 528us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:30.265524 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:30.265709 (+ 185us) server_negotiation.cc:407] Connection header received
0925 02:47:30.266784 (+ 1075us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:30.266788 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:30.267023 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:30.267203 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:47:30.268079 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:30.268955 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:30.270321 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:30.270802 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:30.270961 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:30.273640 (+ 2679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:47:30.273645 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:47:30.273655 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:47:30.273675 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:47:30.275277 (+ 1602us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:47:30.275762 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:47:30.275767 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:47:30.275768 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:47:30.275893 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:47:30.276199 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:47:30.276201 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:47:30.276202 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:47:30.276630 (+ 428us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:47:30.276736 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:30.277245 (+ 509us) server_negotiation.cc:299] Negotiation successful
0925 02:47:30.277450 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":35,"server-negotiator.queue_time_us":385,"thread_start_us":134,"threads_started":1}
I20260925 02:47:30.278350 4023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:30.264976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:43651 (local address 127.0.0.1:51908)
0925 02:47:30.265558 (+ 582us) negotiation.cc:107] Waiting for socket to connect
0925 02:47:30.265607 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0925 02:47:30.266448 (+ 841us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:47:30.267210 (+ 762us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:47:30.267215 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:47:30.267675 (+ 460us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:47:30.267924 (+ 249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:47:30.267939 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:47:30.269091 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:47:30.269095 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:47:30.270188 (+ 1093us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:47:30.270201 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:47:30.270998 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:47:30.271003 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:47:30.271143 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:30.271742 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:47:30.271766 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:47:30.273486 (+ 1720us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:47:30.275387 (+ 1901us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:47:30.275395 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:47:30.275404 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:47:30.275632 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:47:30.276002 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:47:30.276005 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:47:30.276006 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:47:30.276119 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:47:30.276738 (+ 619us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:47:30.276743 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:47:30.276877 (+ 134us) client_negotiation.cc:769] Sending connection context
0925 02:47:30.277144 (+ 267us) client_negotiation.cc:240] Negotiation successful
0925 02:47:30.277515 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":349,"mutex_wait_us":61,"spinlock_wait_cycles":27904,"thread_start_us":171,"threads_started":1}
I20260925 02:47:30.281015 4020 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.157.189" port: 42211 } has no permanent_uuid. Determining permanent_uuid...
I20260925 02:47:30.281729 4023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:30.281439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.189:42211 (local address 127.0.0.1:36578)
0925 02:47:30.281581 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0925 02:47:30.281661 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.189:42211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
W20260925 02:47:30.281908 3970 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.157.189:42211: connect: Connection refused (error 111)
I20260925 02:47:30.282776 4023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:30.282536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.189:42211 (local address 127.0.0.1:36594)
0925 02:47:30.282657 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0925 02:47:30.282711 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.189:42211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260925 02:47:30.282940 4020 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.157.189:42211: Network error: Client connection negotiation failed: client connection to 127.13.157.189:42211: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.157.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.189.keytab.
Entry for principal kudu/127.13.157.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.189.keytab.
I20260925 02:47:30.294747 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790304178325258-13942-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790304178325258-13942-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790304178325258-13942-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790304178325258-13942-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.157.189:42211
--webserver_interface=127.13.157.189
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.189.keytab
--principal=kudu/127.13.157.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:43127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.157.190:43651,127.13.157.189:42211,127.13.157.188:33993
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 02:47:30.332195 4020 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.157.189" port: 42211 } attempt: 1
I20260925 02:47:30.333104 4023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:30.332756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.189:42211 (local address 127.0.0.1:36606)
0925 02:47:30.332931 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0925 02:47:30.333009 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.189:42211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260925 02:47:30.333986 4023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:30.333734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.189:42211 (local address 127.0.0.1:36612)
0925 02:47:30.333849 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0925 02:47:30.333911 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.189:42211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260925 02:47:30.334142 4020 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.157.189:42211: Network error: Client connection negotiation failed: client connection to 127.13.157.189:42211: connect: Connection refused (error 111)
I20260925 02:47:30.402312 4020 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.157.189" port: 42211 } attempt: 2
I20260925 02:47:30.403172 4023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:30.402826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.189:42211 (local address 127.0.0.1:36620)
0925 02:47:30.403014 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0925 02:47:30.403084 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.189:42211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260925 02:47:30.403848 4023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:30.403656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.189:42211 (local address 127.0.0.1:36628)
0925 02:47:30.403746 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0925 02:47:30.403794 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.189:42211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260925 02:47:30.404000 4020 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.157.189:42211: Network error: Client connection negotiation failed: client connection to 127.13.157.189:42211: connect: Connection refused (error 111)
W20260925 02:47:30.404918 4027 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260925 02:47:30.405174 4027 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:47:30.405210 4027 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:47:30.405267 4027 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:47:30.409266 4027 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:47:30.409317 4027 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:47:30.409334 4027 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:47:30.409349 4027 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:47:30.409363 4027 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:47:30.414610 4027 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:43127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790304178325258-13942-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790304178325258-13942-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.13.157.190:43651,127.13.157.189:42211,127.13.157.188:33993
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.189.keytab
--principal=kudu/127.13.157.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.157.189:42211
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790304178325258-13942-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.13.157.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4027
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790304178325258-13942-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:47:30.415679 4027 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:47:30.416676 4027 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:47:30.424854 4035 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:47:30.424876 4032 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:47:30.424897 4033 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:47:30.425478 4027 server_base.cc:1034] running on GCE node
I20260925 02:47:30.426213 4027 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:47:30.427724 4027 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:47:30.428859 4027 hybrid_clock.cc:648] HybridClock initialized: now 1790304450428786 us; error 87 us; skew 500 ppm
Sep 25 02:47:30 dist-test-slave-kznw krb5kdc[3939](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304450, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:47:30.432224 4027 init.cc:377] Logged in from keytab as kudu/127.13.157.189@KRBTEST.COM (short username kudu)
I20260925 02:47:30.433283 4027 webserver.cc:460] Webserver started at http://127.13.157.189:46757/ using document root <none> and password file <none>
I20260925 02:47:30.433935 4027 fs_manager.cc:359] Metadata directory not provided
I20260925 02:47:30.434016 4027 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:47:30.434242 4027 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:47:30.436841 4027 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790304178325258-13942-0/minicluster-data/master-1/data/instance:
uuid: "97953dc98dfc46ff9fcfb0bd3e82281b"
format_stamp: "Formatted at 2026-09-25 02:47:30 on dist-test-slave-kznw"
server_key: "e5f5208c03bac83f0cfe34678996c6da"
server_key_iv: "b22506ee1154d65b47bfc86b106218ff"
server_key_version: "encryptionkey@0"
I20260925 02:47:30.437413 4027 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790304178325258-13942-0/minicluster-data/master-1/wal/instance:
uuid: "97953dc98dfc46ff9fcfb0bd3e82281b"
format_stamp: "Formatted at 2026-09-25 02:47:30 on dist-test-slave-kznw"
server_key: "e5f5208c03bac83f0cfe34678996c6da"
server_key_iv: "b22506ee1154d65b47bfc86b106218ff"
server_key_version: "encryptionkey@0"
I20260925 02:47:30.441210 4027 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260925 02:47:30.443557 4042 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:47:30.444744 4027 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260925 02:47:30.444846 4027 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790304178325258-13942-0/minicluster-data/master-1/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790304178325258-13942-0/minicluster-data/master-1/wal
uuid: "97953dc98dfc46ff9fcfb0bd3e82281b"
format_stamp: "Formatted at 2026-09-25 02:47:30 on dist-test-slave-kznw"
server_key: "e5f5208c03bac83f0cfe34678996c6da"
server_key_iv: "b22506ee1154d65b47bfc86b106218ff"
server_key_version: "encryptionkey@0"
I20260925 02:47:30.444939 4027 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790304178325258-13942-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790304178325258-13942-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790304178325258-13942-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:47:30.454679 4027 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:47:30.457587 4027 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:47:30.457746 4027 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:47:30.464820 4027 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.189:42211
I20260925 02:47:30.465113 4094 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.189:42211 every 8 connection(s)
I20260925 02:47:30.465764 4027 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790304178325258-13942-0/minicluster-data/master-1/data/info.pb
I20260925 02:47:30.468760 4095 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:47:30.470952 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 4027
I20260925 02:47:30.471086 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790304178325258-13942-0/minicluster-data/master-1/wal/instance
I20260925 02:47:30.471349 13942 external_mini_cluster.cc:1428] Setting key cfdf0aa62990e21526d41e4da3bcecf0
I20260925 02:47:30.472968 4095 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.157.190" port: 43651 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.13.157.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.157.188@KRBTEST.COM" created.
Sep 25 02:47:30 dist-test-slave-kznw krb5kdc[3939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304450, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.189@KRBTEST.COM for kudu/127.13.157.190@KRBTEST.COM
I20260925 02:47:30.487110 4022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:30.475051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51914 (local address 127.13.157.190:43651)
0925 02:47:30.475204 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:30.475208 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:30.476110 (+ 902us) server_negotiation.cc:407] Connection header received
0925 02:47:30.477042 (+ 932us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:30.477046 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:30.477109 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:30.477186 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:47:30.478369 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:30.478897 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:30.480107 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:30.480407 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:30.480523 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:30.483080 (+ 2557us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:47:30.483083 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:47:30.483085 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:47:30.483112 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:47:30.484643 (+ 1531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:47:30.485216 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:47:30.485219 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:47:30.485220 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:47:30.485295 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:47:30.485695 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:47:30.485698 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:47:30.485700 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:47:30.485955 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:47:30.486053 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:30.486779 (+ 726us) server_negotiation.cc:299] Negotiation successful
0925 02:47:30.486938 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260925 02:47:30.488008 4098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:30.475442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:43651 (local address 127.0.0.1:51914)
0925 02:47:30.475917 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0925 02:47:30.475960 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0925 02:47:30.476828 (+ 868us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:47:30.477357 (+ 529us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:47:30.477364 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:47:30.477737 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:47:30.478227 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:47:30.478240 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:47:30.479059 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:47:30.479062 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:47:30.479982 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:47:30.479992 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:47:30.480574 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:47:30.480579 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:47:30.480741 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:30.481273 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:47:30.481289 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:47:30.482857 (+ 1568us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:47:30.484761 (+ 1904us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:47:30.484767 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:47:30.484783 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:47:30.485116 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:47:30.485368 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:47:30.485371 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:47:30.485373 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:47:30.485574 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:47:30.486115 (+ 541us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:47:30.486122 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:47:30.486488 (+ 366us) client_negotiation.cc:769] Sending connection context
0925 02:47:30.486705 (+ 217us) client_negotiation.cc:240] Negotiation successful
0925 02:47:30.487106 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":113,"threads_started":1}
I20260925 02:47:30.489509 4095 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.157.189" port: 42211 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 02:47:30 dist-test-slave-kznw krb5kdc[3939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304450, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.189@KRBTEST.COM for kudu/127.13.157.189@KRBTEST.COM
I20260925 02:47:30.495146 4020 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.157.189" port: 42211 } attempt: 3
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.157.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.188.keytab.
Entry for principal kudu/127.13.157.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.188.keytab.
I20260925 02:47:30.498486 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790304178325258-13942-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790304178325258-13942-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790304178325258-13942-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790304178325258-13942-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.157.188:33993
--webserver_interface=127.13.157.188
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.188.keytab
--principal=kudu/127.13.157.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:43127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.157.190:43651,127.13.157.189:42211,127.13.157.188:33993
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 02:47:30.499858 4098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:30.490050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.189:42211 (local address 127.0.0.1:36644)
0925 02:47:30.490550 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0925 02:47:30.490562 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 02:47:30.490658 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:47:30.491050 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:47:30.491052 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:47:30.491187 (+ 135us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:47:30.491289 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:47:30.491294 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:47:30.492387 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:47:30.492391 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:47:30.492969 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:47:30.492977 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:47:30.493574 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:47:30.493577 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:47:30.493655 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:30.494064 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:47:30.494075 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:47:30.495605 (+ 1530us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:47:30.497992 (+ 2387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:47:30.497996 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:47:30.497998 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:47:30.498254 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:47:30.498697 (+ 443us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:47:30.498700 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:47:30.498702 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:47:30.498793 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:47:30.499339 (+ 546us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:47:30.499343 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:47:30.499503 (+ 160us) client_negotiation.cc:769] Sending connection context
0925 02:47:30.499608 (+ 105us) client_negotiation.cc:240] Negotiation successful
0925 02:47:30.499730 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":436,"mutex_wait_us":1}
I20260925 02:47:30.500499 4100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:30.490242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36644 (local address 127.13.157.189:42211)
0925 02:47:30.490563 (+ 321us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:30.490571 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:30.490614 (+ 43us) server_negotiation.cc:407] Connection header received
0925 02:47:30.490736 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:30.490741 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:30.490929 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:30.491049 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:47:30.491404 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:30.492279 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:30.493069 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:30.493470 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:30.493560 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:30.496045 (+ 2485us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:47:30.496054 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:47:30.496065 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:47:30.496086 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:47:30.497882 (+ 1796us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:47:30.498371 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:47:30.498378 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:47:30.498379 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:47:30.498471 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:47:30.498885 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:47:30.498888 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:47:30.498889 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:47:30.499238 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:47:30.499370 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:30.500067 (+ 697us) server_negotiation.cc:299] Negotiation successful
0925 02:47:30.500197 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":223,"thread_start_us":129,"threads_started":1}
Sep 25 02:47:30 dist-test-slave-kznw krb5kdc[3939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304450, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.190@KRBTEST.COM for kudu/127.13.157.189@KRBTEST.COM
I20260925 02:47:30.502655 4095 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.157.188" port: 33993 } has no permanent_uuid. Determining permanent_uuid...
I20260925 02:47:30.503460 4098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:30.503134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.188:33993 (local address 127.0.0.1:32856)
0925 02:47:30.503291 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0925 02:47:30.503359 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.188:33993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
W20260925 02:47:30.503670 4044 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.157.188:33993: connect: Connection refused (error 111)
I20260925 02:47:30.504617 4098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:30.504351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.188:33993 (local address 127.0.0.1:32866)
0925 02:47:30.504501 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0925 02:47:30.504559 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.188:33993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260925 02:47:30.504679 4023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:30.495616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.189:42211 (local address 127.0.0.1:36658)
0925 02:47:30.495779 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0925 02:47:30.495796 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 02:47:30.495912 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:47:30.496307 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:47:30.496311 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:47:30.496574 (+ 263us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:47:30.496680 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:47:30.496686 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:47:30.497805 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:47:30.497809 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:47:30.499122 (+ 1313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:47:30.499133 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:47:30.499651 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:47:30.499655 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:47:30.499765 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:30.500216 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:47:30.500233 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:47:30.501482 (+ 1249us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:47:30.503204 (+ 1722us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:47:30.503207 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:47:30.503209 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:47:30.503398 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:47:30.503668 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:47:30.503671 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:47:30.503673 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:47:30.503813 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:47:30.504216 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:47:30.504219 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:47:30.504323 (+ 104us) client_negotiation.cc:769] Sending connection context
0925 02:47:30.504392 (+ 69us) client_negotiation.cc:240] Negotiation successful
0925 02:47:30.504531 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
W20260925 02:47:30.504782 4095 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.157.188:33993: Network error: Client connection negotiation failed: client connection to 127.13.157.188:33993: connect: Connection refused (error 111)
I20260925 02:47:30.505203 4101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:30.495679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36658 (local address 127.13.157.189:42211)
0925 02:47:30.496094 (+ 415us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:30.496101 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:30.496112 (+ 11us) server_negotiation.cc:407] Connection header received
0925 02:47:30.496139 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:30.496141 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:30.496173 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:30.496243 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:47:30.496797 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:30.497674 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:30.499219 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:30.499529 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:30.499628 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:30.501639 (+ 2011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:47:30.501643 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:47:30.501645 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:47:30.501670 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:47:30.503111 (+ 1441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:47:30.503507 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:47:30.503511 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:47:30.503512 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:47:30.503593 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:47:30.503896 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:47:30.503899 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:47:30.503908 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:47:30.504104 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:47:30.504936 (+ 832us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:30.505008 (+ 72us) server_negotiation.cc:299] Negotiation successful
0925 02:47:30.505095 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":344,"thread_start_us":145,"threads_started":1}
I20260925 02:47:30.505781 4020 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.157.188" port: 33993 } has no permanent_uuid. Determining permanent_uuid...
I20260925 02:47:30.506407 4023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:30.506199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.188:33993 (local address 127.0.0.1:32874)
0925 02:47:30.506302 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0925 02:47:30.506349 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.188:33993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260925 02:47:30.507148 4023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:30.506934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.188:33993 (local address 127.0.0.1:32888)
0925 02:47:30.507051 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0925 02:47:30.507099 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.188:33993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260925 02:47:30.507272 4020 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.157.188:33993: Network error: Client connection negotiation failed: client connection to 127.13.157.188:33993: connect: Connection refused (error 111)
I20260925 02:47:30.538468 4020 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.157.188" port: 33993 } attempt: 1
I20260925 02:47:30.539448 4023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:30.539040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.188:33993 (local address 127.0.0.1:32894)
0925 02:47:30.539245 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0925 02:47:30.539337 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.188:33993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260925 02:47:30.540261 4023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:30.540021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.188:33993 (local address 127.0.0.1:32896)
0925 02:47:30.540128 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0925 02:47:30.540189 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.188:33993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260925 02:47:30.540469 4020 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.157.188:33993: Network error: Client connection negotiation failed: client connection to 127.13.157.188:33993: connect: Connection refused (error 111)
I20260925 02:47:30.554059 4095 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.157.188" port: 33993 } attempt: 1
I20260925 02:47:30.554988 4098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:30.554664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.188:33993 (local address 127.0.0.1:32912)
0925 02:47:30.554808 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0925 02:47:30.554870 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.188:33993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260925 02:47:30.555759 4098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:30.555518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.188:33993 (local address 127.0.0.1:32922)
0925 02:47:30.555648 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0925 02:47:30.555700 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.188:33993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260925 02:47:30.555894 4095 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.157.188:33993: Network error: Client connection negotiation failed: client connection to 127.13.157.188:33993: connect: Connection refused (error 111)
W20260925 02:47:30.613046 4102 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260925 02:47:30.613323 4102 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:47:30.613358 4102 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:47:30.613417 4102 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260925 02:47:30.615623 4020 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.157.188" port: 33993 } attempt: 2
I20260925 02:47:30.616456 4023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:30.616107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.188:33993 (local address 127.0.0.1:32936)
0925 02:47:30.616280 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0925 02:47:30.616358 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.188:33993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260925 02:47:30.617239 4023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:30.617037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.188:33993 (local address 127.0.0.1:32944)
0925 02:47:30.617132 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0925 02:47:30.617180 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.188:33993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260925 02:47:30.617390 4020 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.157.188:33993: Network error: Client connection negotiation failed: client connection to 127.13.157.188:33993: connect: Connection refused (error 111)
W20260925 02:47:30.617669 4102 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:47:30.617717 4102 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:47:30.617748 4102 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:47:30.617777 4102 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:47:30.617805 4102 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:47:30.623435 4102 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:43127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790304178325258-13942-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790304178325258-13942-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.13.157.190:43651,127.13.157.189:42211,127.13.157.188:33993
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.188.keytab
--principal=kudu/127.13.157.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.157.188:33993
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790304178325258-13942-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.13.157.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4102
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790304178325258-13942-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:47:30.624030 4095 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.157.188" port: 33993 } attempt: 2
I20260925 02:47:30.624714 4102 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:47:30.624781 4098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:30.624510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.188:33993 (local address 127.0.0.1:32952)
0925 02:47:30.624644 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0925 02:47:30.624707 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.188:33993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260925 02:47:30.625523 4098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:30.625281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.188:33993 (local address 127.0.0.1:32966)
0925 02:47:30.625394 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0925 02:47:30.625450 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.188:33993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260925 02:47:30.625747 4095 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.157.188:33993: Network error: Client connection negotiation failed: client connection to 127.13.157.188:33993: connect: Connection refused (error 111)
I20260925 02:47:30.625941 4102 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:47:30.634321 4108 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:47:30.634356 4102 server_base.cc:1034] running on GCE node
W20260925 02:47:30.634315 4109 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:47:30.634318 4111 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:47:30.635123 4102 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:47:30.636246 4102 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:47:30.637411 4102 hybrid_clock.cc:648] HybridClock initialized: now 1790304450637378 us; error 41 us; skew 500 ppm
Sep 25 02:47:30 dist-test-slave-kznw krb5kdc[3939](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304450, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:47:30.640147 4102 init.cc:377] Logged in from keytab as kudu/127.13.157.188@KRBTEST.COM (short username kudu)
I20260925 02:47:30.641050 4102 webserver.cc:460] Webserver started at http://127.13.157.188:36621/ using document root <none> and password file <none>
I20260925 02:47:30.641592 4102 fs_manager.cc:359] Metadata directory not provided
I20260925 02:47:30.641664 4102 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:47:30.641861 4102 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:47:30.643987 4102 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790304178325258-13942-0/minicluster-data/master-2/data/instance:
uuid: "a48e29845e1e45ea8488399b9424a611"
format_stamp: "Formatted at 2026-09-25 02:47:30 on dist-test-slave-kznw"
server_key: "c16128ba875c7901b58e549baa28cb0a"
server_key_iv: "82a3acef17fbcbe0ee6cf968e9aa84d6"
server_key_version: "encryptionkey@0"
I20260925 02:47:30.644554 4102 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790304178325258-13942-0/minicluster-data/master-2/wal/instance:
uuid: "a48e29845e1e45ea8488399b9424a611"
format_stamp: "Formatted at 2026-09-25 02:47:30 on dist-test-slave-kznw"
server_key: "c16128ba875c7901b58e549baa28cb0a"
server_key_iv: "82a3acef17fbcbe0ee6cf968e9aa84d6"
server_key_version: "encryptionkey@0"
I20260925 02:47:30.648332 4102 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260925 02:47:30.650949 4118 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:47:30.652195 4102 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260925 02:47:30.652328 4102 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790304178325258-13942-0/minicluster-data/master-2/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790304178325258-13942-0/minicluster-data/master-2/wal
uuid: "a48e29845e1e45ea8488399b9424a611"
format_stamp: "Formatted at 2026-09-25 02:47:30 on dist-test-slave-kznw"
server_key: "c16128ba875c7901b58e549baa28cb0a"
server_key_iv: "82a3acef17fbcbe0ee6cf968e9aa84d6"
server_key_version: "encryptionkey@0"
I20260925 02:47:30.652427 4102 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790304178325258-13942-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790304178325258-13942-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790304178325258-13942-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:47:30.668498 4102 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:47:30.671506 4102 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:47:30.671684 4102 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:47:30.678310 4102 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.188:33993
I20260925 02:47:30.678551 4171 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.188:33993 every 8 connection(s)
I20260925 02:47:30.679322 4102 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790304178325258-13942-0/minicluster-data/master-2/data/info.pb
I20260925 02:47:30.682030 4172 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:47:30.685719 4172 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.157.190" port: 43651 } has no permanent_uuid. Determining permanent_uuid...
I20260925 02:47:30.686816 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 4102
I20260925 02:47:30.686990 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790304178325258-13942-0/minicluster-data/master-2/wal/instance
I20260925 02:47:30.687348 13942 external_mini_cluster.cc:1428] Setting key eb4b0290ad76532b9fa47eb18002e120
Sep 25 02:47:30 dist-test-slave-kznw krb5kdc[3939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304450, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.188@KRBTEST.COM for kudu/127.13.157.190@KRBTEST.COM
Sep 25 02:47:30 dist-test-slave-kznw krb5kdc[3939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304450, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.157.188@KRBTEST.COM
I20260925 02:47:30.700027 4022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:30.687656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51918 (local address 127.13.157.190:43651)
0925 02:47:30.687788 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:30.687792 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:30.688857 (+ 1065us) server_negotiation.cc:407] Connection header received
0925 02:47:30.689959 (+ 1102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:30.689963 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:30.690018 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:30.690096 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:47:30.691264 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:30.692051 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:30.693158 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:30.693452 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:30.693564 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:30.696181 (+ 2617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:47:30.696184 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:47:30.696186 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:47:30.696213 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:47:30.697714 (+ 1501us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:47:30.698260 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:47:30.698264 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:47:30.698266 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:47:30.698354 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:47:30.698756 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:47:30.698760 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:47:30.698761 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:47:30.699116 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:47:30.699204 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:30.699736 (+ 532us) server_negotiation.cc:299] Negotiation successful
0925 02:47:30.699851 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260925 02:47:30.700922 4175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:30.688180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:43651 (local address 127.0.0.1:51918)
0925 02:47:30.688694 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0925 02:47:30.688748 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0925 02:47:30.689707 (+ 959us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:47:30.690185 (+ 478us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:47:30.690191 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:47:30.690730 (+ 539us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:47:30.691113 (+ 383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:47:30.691130 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:47:30.692188 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:47:30.692192 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:47:30.693025 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:47:30.693036 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:47:30.693618 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:47:30.693622 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:47:30.693774 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:30.694382 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:47:30.694404 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:47:30.696004 (+ 1600us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:47:30.697832 (+ 1828us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:47:30.697841 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:47:30.697849 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:47:30.698166 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:47:30.698441 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:47:30.698445 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:47:30.698447 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:47:30.698623 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:47:30.699221 (+ 598us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:47:30.699228 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:47:30.699501 (+ 273us) client_negotiation.cc:769] Sending connection context
0925 02:47:30.699669 (+ 168us) client_negotiation.cc:240] Negotiation successful
0925 02:47:30.699992 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"spinlock_wait_cycles":768,"thread_start_us":133,"threads_started":1}
I20260925 02:47:30.700927 4176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:30.689092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32976 (local address 127.13.157.188:33993)
0925 02:47:30.689451 (+ 359us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:30.689461 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:30.689496 (+ 35us) server_negotiation.cc:407] Connection header received
0925 02:47:30.689711 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:30.689726 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:30.689940 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:30.690132 (+ 192us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:47:30.691121 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:30.692410 (+ 1289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:30.693263 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:30.693638 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:30.693779 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:30.696603 (+ 2824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:47:30.696610 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:47:30.696622 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:47:30.696643 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:47:30.698210 (+ 1567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:47:30.698764 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:47:30.698770 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:47:30.698772 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:47:30.698853 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:47:30.699214 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:47:30.699217 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:47:30.699219 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:47:30.699527 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:47:30.699694 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:30.699923 (+ 229us) server_negotiation.cc:299] Negotiation successful
0925 02:47:30.700055 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":206,"spinlock_wait_cycles":1280,"thread_start_us":99,"threads_started":1}
I20260925 02:47:30.703807 4172 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.157.189" port: 42211 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 02:47:30 dist-test-slave-kznw krb5kdc[3939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304450, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.157.190@KRBTEST.COM
Sep 25 02:47:30 dist-test-slave-kznw krb5kdc[3939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304450, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.188@KRBTEST.COM for kudu/127.13.157.189@KRBTEST.COM
I20260925 02:47:30.712193 4022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:30.703339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51922 (local address 127.13.157.190:43651)
0925 02:47:30.703505 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:30.703508 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:30.703547 (+ 39us) server_negotiation.cc:407] Connection header received
0925 02:47:30.703665 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:30.703667 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:30.703702 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:30.703767 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:47:30.704504 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:30.705162 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:30.706163 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:30.706498 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:30.706594 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:30.709074 (+ 2480us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:47:30.709078 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:47:30.709081 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:47:30.709104 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:47:30.710474 (+ 1370us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:47:30.710942 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:47:30.710946 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:47:30.710947 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:47:30.711045 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:47:30.711282 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:47:30.711285 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:47:30.711286 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:47:30.711478 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:47:30.711710 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:30.711810 (+ 100us) server_negotiation.cc:299] Negotiation successful
0925 02:47:30.711896 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260925 02:47:30.712396 4175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:30.704246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.189:42211 (local address 127.0.0.1:36668)
0925 02:47:30.704388 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0925 02:47:30.704400 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 02:47:30.704459 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:47:30.704676 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:47:30.704679 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:47:30.704832 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:47:30.704936 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:47:30.704942 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:47:30.705937 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:47:30.705939 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:47:30.706645 (+ 706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:47:30.706654 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:47:30.707231 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:47:30.707234 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:47:30.707309 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:30.707733 (+ 424us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:47:30.707743 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:47:30.709288 (+ 1545us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:47:30.710932 (+ 1644us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:47:30.710936 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:47:30.710938 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:47:30.711180 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:47:30.711473 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:47:30.711476 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:47:30.711478 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:47:30.711566 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:47:30.711953 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:47:30.711956 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:47:30.712047 (+ 91us) client_negotiation.cc:769] Sending connection context
0925 02:47:30.712146 (+ 99us) client_negotiation.cc:240] Negotiation successful
0925 02:47:30.712262 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":88}
I20260925 02:47:30.712459 4101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:30.704362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36668 (local address 127.13.157.189:42211)
0925 02:47:30.704493 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:30.704496 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:30.704508 (+ 12us) server_negotiation.cc:407] Connection header received
0925 02:47:30.704545 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:30.704547 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:30.704584 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:30.704644 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:47:30.705055 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:30.705829 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:30.706768 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:30.707125 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:30.707236 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:30.709417 (+ 2181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:47:30.709421 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:47:30.709423 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:47:30.709450 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:47:30.710817 (+ 1367us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:47:30.711262 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:47:30.711266 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:47:30.711268 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:47:30.711346 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:47:30.711650 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:47:30.711653 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:47:30.711655 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:47:30.711858 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:47:30.711943 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:30.712196 (+ 253us) server_negotiation.cc:299] Negotiation successful
0925 02:47:30.712312 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260925 02:47:30.713347 4172 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.157.188" port: 33993 } has no permanent_uuid. Determining permanent_uuid...
I20260925 02:47:30.716575 4020 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.157.188" port: 33993 } attempt: 3
I20260925 02:47:30.716905 4095 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.157.188" port: 33993 } attempt: 3
Sep 25 02:47:30 dist-test-slave-kznw krb5kdc[3939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304450, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.188@KRBTEST.COM for kudu/127.13.157.188@KRBTEST.COM
Sep 25 02:47:30 dist-test-slave-kznw krb5kdc[3939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304450, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.157.189@KRBTEST.COM
Sep 25 02:47:30 dist-test-slave-kznw krb5kdc[3939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304450, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.190@KRBTEST.COM for kudu/127.13.157.188@KRBTEST.COM
I20260925 02:47:30.722682 4175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:30.713917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.188:33993 (local address 127.0.0.1:32990)
0925 02:47:30.713998 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0925 02:47:30.714010 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 02:47:30.714073 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:47:30.714288 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:47:30.714290 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:47:30.714404 (+ 114us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:47:30.714475 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:47:30.714479 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:47:30.715278 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:47:30.715281 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:47:30.715885 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:47:30.715894 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:47:30.716419 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:47:30.716423 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:47:30.716501 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:30.716890 (+ 389us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:47:30.716905 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:47:30.718539 (+ 1634us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:47:30.721069 (+ 2530us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:47:30.721071 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:47:30.721073 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:47:30.721288 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:47:30.721768 (+ 480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:47:30.721771 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:47:30.721773 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:47:30.721850 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:47:30.722251 (+ 401us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:47:30.722254 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:47:30.722329 (+ 75us) client_negotiation.cc:769] Sending connection context
0925 02:47:30.722404 (+ 75us) client_negotiation.cc:240] Negotiation successful
0925 02:47:30.722529 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24}
Sep 25 02:47:30 dist-test-slave-kznw krb5kdc[3939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304450, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.189@KRBTEST.COM for kudu/127.13.157.188@KRBTEST.COM
I20260925 02:47:30.723233 4176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:30.713988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32990 (local address 127.13.157.188:33993)
0925 02:47:30.714096 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:30.714100 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:30.714130 (+ 30us) server_negotiation.cc:407] Connection header received
0925 02:47:30.714162 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:30.714165 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:30.714200 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:30.714270 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:47:30.714571 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:30.715181 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:30.716001 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:30.716292 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:30.716385 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:30.718874 (+ 2489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:47:30.718877 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:47:30.718880 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:47:30.718928 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:47:30.720965 (+ 2037us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:47:30.721605 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:47:30.721607 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:47:30.721609 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:47:30.721688 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:47:30.721950 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:47:30.721954 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:47:30.721956 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:47:30.722162 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:47:30.722939 (+ 777us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:30.723010 (+ 71us) server_negotiation.cc:299] Negotiation successful
0925 02:47:30.723104 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260925 02:47:30.723488 4101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:30.713460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36682 (local address 127.13.157.189:42211)
0925 02:47:30.713586 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:30.713590 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:30.713603 (+ 13us) server_negotiation.cc:407] Connection header received
0925 02:47:30.713636 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:30.713639 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:30.713675 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:30.713734 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:47:30.714390 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:30.714850 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:30.715724 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:30.716050 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:30.716121 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:30.718846 (+ 2725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:47:30.718850 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:47:30.718853 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:47:30.718875 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:47:30.720904 (+ 2029us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:47:30.721354 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:47:30.721357 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:47:30.721359 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:47:30.721439 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:47:30.722232 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:47:30.722236 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:47:30.722237 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:47:30.722428 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:47:30.722622 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:30.723245 (+ 623us) server_negotiation.cc:299] Negotiation successful
0925 02:47:30.723357 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260925 02:47:30.726548 4172 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a48e29845e1e45ea8488399b9424a611: Bootstrap starting.
I20260925 02:47:30.727285 13942 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 02:47:30.728709 4098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:30.717217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.188:33993 (local address 127.0.0.1:33012)
0925 02:47:30.717384 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0925 02:47:30.717397 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 02:47:30.717460 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:47:30.718893 (+ 1433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:47:30.718895 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:47:30.719168 (+ 273us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:47:30.719253 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:47:30.719260 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:47:30.720240 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:47:30.720245 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:47:30.721038 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:47:30.721048 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:47:30.721600 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:47:30.721603 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:47:30.721689 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:30.722077 (+ 388us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:47:30.722091 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:47:30.723742 (+ 1651us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:47:30.726303 (+ 2561us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:47:30.726306 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:47:30.726308 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:47:30.726530 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:47:30.726852 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:47:30.726855 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:47:30.726856 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:47:30.726989 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:47:30.728267 (+ 1278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:47:30.728270 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:47:30.728357 (+ 87us) client_negotiation.cc:769] Sending connection context
0925 02:47:30.728434 (+ 77us) client_negotiation.cc:240] Negotiation successful
0925 02:47:30.728558 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93}
I20260925 02:47:30.728725 4023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:30.717020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.188:33993 (local address 127.0.0.1:33000)
0925 02:47:30.717131 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0925 02:47:30.717145 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 02:47:30.717210 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:47:30.717902 (+ 692us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:47:30.717905 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:47:30.718059 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:47:30.718132 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:47:30.718137 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:47:30.719655 (+ 1518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:47:30.719658 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:47:30.720454 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:47:30.720465 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:47:30.721077 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:47:30.721081 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:47:30.721163 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:30.721536 (+ 373us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:47:30.721550 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:47:30.723329 (+ 1779us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:47:30.725956 (+ 2627us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:47:30.725959 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:47:30.725961 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:47:30.726194 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:47:30.726670 (+ 476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:47:30.726673 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:47:30.726675 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:47:30.726761 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:47:30.728302 (+ 1541us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:47:30.728305 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:47:30.728389 (+ 84us) client_negotiation.cc:769] Sending connection context
0925 02:47:30.728477 (+ 88us) client_negotiation.cc:240] Negotiation successful
0925 02:47:30.728591 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260925 02:47:30.729212 4177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:30.717244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33000 (local address 127.13.157.188:33993)
0925 02:47:30.717710 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:30.717718 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:30.717731 (+ 13us) server_negotiation.cc:407] Connection header received
0925 02:47:30.717762 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:30.717765 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:30.717803 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:30.717882 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:47:30.718231 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:30.719559 (+ 1328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:30.720587 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:30.720949 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:30.721348 (+ 399us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:30.723562 (+ 2214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:47:30.723566 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:47:30.723568 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:47:30.723591 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:47:30.725836 (+ 2245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:47:30.726490 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:47:30.726493 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:47:30.726495 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:47:30.726571 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:47:30.727174 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:47:30.727176 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:47:30.727178 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:47:30.728203 (+ 1025us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:47:30.728869 (+ 666us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:30.728950 (+ 81us) server_negotiation.cc:299] Negotiation successful
0925 02:47:30.729058 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":405,"thread_start_us":101,"threads_started":1}
I20260925 02:47:30.729225 4172 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a48e29845e1e45ea8488399b9424a611: Neither blocks nor log segments found. Creating new log.
I20260925 02:47:30.728446 3985 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "55a954a30ce64942959f12b90c70cae4"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51922
I20260925 02:47:30.730166 4172 log.cc:824] T 00000000000000000000000000000000 P a48e29845e1e45ea8488399b9424a611: Log is configured to *not* fsync() on all Append() calls
I20260925 02:47:30.731748 4020 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55a954a30ce64942959f12b90c70cae4: Bootstrap starting.
I20260925 02:47:30.732411 4172 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a48e29845e1e45ea8488399b9424a611: No bootstrap required, opened a new log
I20260925 02:47:30.733973 4020 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 55a954a30ce64942959f12b90c70cae4: Neither blocks nor log segments found. Creating new log.
I20260925 02:47:30.734664 4020 log.cc:824] T 00000000000000000000000000000000 P 55a954a30ce64942959f12b90c70cae4: Log is configured to *not* fsync() on all Append() calls
I20260925 02:47:30.735479 4178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:30.717449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33012 (local address 127.13.157.188:33993)
0925 02:47:30.718715 (+ 1266us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:30.718720 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:30.718728 (+ 8us) server_negotiation.cc:407] Connection header received
0925 02:47:30.718748 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:30.718749 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:30.718780 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:30.719251 (+ 471us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:47:30.719365 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:30.720117 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:30.721161 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:30.721500 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:30.721601 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:30.724297 (+ 2696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:47:30.724301 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:47:30.724303 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:47:30.724327 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:47:30.726195 (+ 1868us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:47:30.726684 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:47:30.726687 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:47:30.726689 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:47:30.726751 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:47:30.727103 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:47:30.727107 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:47:30.727108 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:47:30.728149 (+ 1041us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:47:30.729511 (+ 1362us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:30.735136 (+ 5625us) server_negotiation.cc:299] Negotiation successful
0925 02:47:30.735295 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1211,"spinlock_wait_cycles":1910528,"thread_start_us":83,"threads_started":1}
I20260925 02:47:30.737167 4020 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55a954a30ce64942959f12b90c70cae4: No bootstrap required, opened a new log
I20260925 02:47:30.737023 4172 raft_consensus.cc:348] T 00000000000000000000000000000000 P a48e29845e1e45ea8488399b9424a611 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55a954a30ce64942959f12b90c70cae4" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 43651 } } peers { permanent_uuid: "97953dc98dfc46ff9fcfb0bd3e82281b" member_type: VOTER last_known_addr { host: "127.13.157.189" port: 42211 } } peers { permanent_uuid: "a48e29845e1e45ea8488399b9424a611" member_type: VOTER last_known_addr { host: "127.13.157.188" port: 33993 } }
I20260925 02:47:30.737262 4172 raft_consensus.cc:729] T 00000000000000000000000000000000 P a48e29845e1e45ea8488399b9424a611 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a48e29845e1e45ea8488399b9424a611, State: Initialized, Role: FOLLOWER
I20260925 02:47:30.737605 4172 consensus_queue.cc:260] T 00000000000000000000000000000000 P a48e29845e1e45ea8488399b9424a611 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55a954a30ce64942959f12b90c70cae4" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 43651 } } peers { permanent_uuid: "97953dc98dfc46ff9fcfb0bd3e82281b" member_type: VOTER last_known_addr { host: "127.13.157.189" port: 42211 } } peers { permanent_uuid: "a48e29845e1e45ea8488399b9424a611" member_type: VOTER last_known_addr { host: "127.13.157.188" port: 33993 } }
I20260925 02:47:30.738128 4095 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97953dc98dfc46ff9fcfb0bd3e82281b: Bootstrap starting.
I20260925 02:47:30.738114 4181 sys_catalog.cc:455] T 00000000000000000000000000000000 P a48e29845e1e45ea8488399b9424a611 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55a954a30ce64942959f12b90c70cae4" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 43651 } } peers { permanent_uuid: "97953dc98dfc46ff9fcfb0bd3e82281b" member_type: VOTER last_known_addr { host: "127.13.157.189" port: 42211 } } peers { permanent_uuid: "a48e29845e1e45ea8488399b9424a611" member_type: VOTER last_known_addr { host: "127.13.157.188" port: 33993 } } }
I20260925 02:47:30.738479 4181 sys_catalog.cc:458] T 00000000000000000000000000000000 P a48e29845e1e45ea8488399b9424a611 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 02:47:30.739014 4172 sys_catalog.cc:564] T 00000000000000000000000000000000 P a48e29845e1e45ea8488399b9424a611 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:47:30.740278 4020 raft_consensus.cc:348] T 00000000000000000000000000000000 P 55a954a30ce64942959f12b90c70cae4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55a954a30ce64942959f12b90c70cae4" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 43651 } } peers { permanent_uuid: "97953dc98dfc46ff9fcfb0bd3e82281b" member_type: VOTER last_known_addr { host: "127.13.157.189" port: 42211 } } peers { permanent_uuid: "a48e29845e1e45ea8488399b9424a611" member_type: VOTER last_known_addr { host: "127.13.157.188" port: 33993 } }
I20260925 02:47:30.740506 4020 raft_consensus.cc:729] T 00000000000000000000000000000000 P 55a954a30ce64942959f12b90c70cae4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55a954a30ce64942959f12b90c70cae4, State: Initialized, Role: FOLLOWER
I20260925 02:47:30.740554 4095 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 97953dc98dfc46ff9fcfb0bd3e82281b: Neither blocks nor log segments found. Creating new log.
I20260925 02:47:30.740859 4020 consensus_queue.cc:260] T 00000000000000000000000000000000 P 55a954a30ce64942959f12b90c70cae4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55a954a30ce64942959f12b90c70cae4" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 43651 } } peers { permanent_uuid: "97953dc98dfc46ff9fcfb0bd3e82281b" member_type: VOTER last_known_addr { host: "127.13.157.189" port: 42211 } } peers { permanent_uuid: "a48e29845e1e45ea8488399b9424a611" member_type: VOTER last_known_addr { host: "127.13.157.188" port: 33993 } }
I20260925 02:47:30.741662 4095 log.cc:824] T 00000000000000000000000000000000 P 97953dc98dfc46ff9fcfb0bd3e82281b: Log is configured to *not* fsync() on all Append() calls
I20260925 02:47:30.742153 4020 sys_catalog.cc:564] T 00000000000000000000000000000000 P 55a954a30ce64942959f12b90c70cae4 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:47:30.742697 3985 raft_consensus.cc:482] T 00000000000000000000000000000000 P 55a954a30ce64942959f12b90c70cae4 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260925 02:47:30.742810 3985 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 55a954a30ce64942959f12b90c70cae4 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:47:30.744182 3985 raft_consensus.cc:504] T 00000000000000000000000000000000 P 55a954a30ce64942959f12b90c70cae4 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55a954a30ce64942959f12b90c70cae4" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 43651 } } peers { permanent_uuid: "97953dc98dfc46ff9fcfb0bd3e82281b" member_type: VOTER last_known_addr { host: "127.13.157.189" port: 42211 } } peers { permanent_uuid: "a48e29845e1e45ea8488399b9424a611" member_type: VOTER last_known_addr { host: "127.13.157.188" port: 33993 } }
W20260925 02:47:30.744391 4192 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a48e29845e1e45ea8488399b9424a611: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 02:47:30.744503 4192 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 02:47:30.742774 4102 master_runner.cc:186] Error getting master registration for 127.13.157.190:43651: OK, instance_id { permanent_uuid: "55a954a30ce64942959f12b90c70cae4" instance_seqno: 1790304450252865 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260925 02:47:30.745918 4102 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260925 02:47:30.745985 4146 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "55a954a30ce64942959f12b90c70cae4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "a48e29845e1e45ea8488399b9424a611"
I20260925 02:47:30.746210 4146 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a48e29845e1e45ea8488399b9424a611 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:47:30.747560 4146 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a48e29845e1e45ea8488399b9424a611 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 55a954a30ce64942959f12b90c70cae4 in term 1.
I20260925 02:47:30.747583 4095 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97953dc98dfc46ff9fcfb0bd3e82281b: No bootstrap required, opened a new log
I20260925 02:47:30.748051 3969 leader_election.cc:302] T 00000000000000000000000000000000 P 55a954a30ce64942959f12b90c70cae4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55a954a30ce64942959f12b90c70cae4, a48e29845e1e45ea8488399b9424a611; no voters:
I20260925 02:47:30.748188 3985 leader_election.cc:288] T 00000000000000000000000000000000 P 55a954a30ce64942959f12b90c70cae4 [CANDIDATE]: Term 1 election: Requested vote from peers 97953dc98dfc46ff9fcfb0bd3e82281b (127.13.157.189:42211), a48e29845e1e45ea8488399b9424a611 (127.13.157.188:33993)
W20260925 02:47:30.748811 3973 tablet.cc:2369] T 00000000000000000000000000000000 P 55a954a30ce64942959f12b90c70cae4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 02:47:30.751941 4095 raft_consensus.cc:348] T 00000000000000000000000000000000 P 97953dc98dfc46ff9fcfb0bd3e82281b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55a954a30ce64942959f12b90c70cae4" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 43651 } } peers { permanent_uuid: "97953dc98dfc46ff9fcfb0bd3e82281b" member_type: VOTER last_known_addr { host: "127.13.157.189" port: 42211 } } peers { permanent_uuid: "a48e29845e1e45ea8488399b9424a611" member_type: VOTER last_known_addr { host: "127.13.157.188" port: 33993 } }
I20260925 02:47:30.752202 4095 raft_consensus.cc:729] T 00000000000000000000000000000000 P 97953dc98dfc46ff9fcfb0bd3e82281b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97953dc98dfc46ff9fcfb0bd3e82281b, State: Initialized, Role: FOLLOWER
I20260925 02:47:30.752553 4095 consensus_queue.cc:260] T 00000000000000000000000000000000 P 97953dc98dfc46ff9fcfb0bd3e82281b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55a954a30ce64942959f12b90c70cae4" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 43651 } } peers { permanent_uuid: "97953dc98dfc46ff9fcfb0bd3e82281b" member_type: VOTER last_known_addr { host: "127.13.157.189" port: 42211 } } peers { permanent_uuid: "a48e29845e1e45ea8488399b9424a611" member_type: VOTER last_known_addr { host: "127.13.157.188" port: 33993 } }
I20260925 02:47:30.754027 4095 sys_catalog.cc:564] T 00000000000000000000000000000000 P 97953dc98dfc46ff9fcfb0bd3e82281b [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:47:30.754971 4207 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55a954a30ce64942959f12b90c70cae4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55a954a30ce64942959f12b90c70cae4" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 43651 } } peers { permanent_uuid: "97953dc98dfc46ff9fcfb0bd3e82281b" member_type: VOTER last_known_addr { host: "127.13.157.189" port: 42211 } } peers { permanent_uuid: "a48e29845e1e45ea8488399b9424a611" member_type: VOTER last_known_addr { host: "127.13.157.188" port: 33993 } } }
I20260925 02:47:30.755102 4207 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55a954a30ce64942959f12b90c70cae4 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 02:47:30.755191 4207 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 55a954a30ce64942959f12b90c70cae4 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:47:30.755264 4207 raft_consensus.cc:686] T 00000000000000000000000000000000 P 55a954a30ce64942959f12b90c70cae4 [term 1 LEADER]: Becoming Leader. State: Replica: 55a954a30ce64942959f12b90c70cae4, State: Running, Role: LEADER
I20260925 02:47:30.755649 4207 consensus_queue.cc:237] T 00000000000000000000000000000000 P 55a954a30ce64942959f12b90c70cae4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55a954a30ce64942959f12b90c70cae4" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 43651 } } peers { permanent_uuid: "97953dc98dfc46ff9fcfb0bd3e82281b" member_type: VOTER last_known_addr { host: "127.13.157.189" port: 42211 } } peers { permanent_uuid: "a48e29845e1e45ea8488399b9424a611" member_type: VOTER last_known_addr { host: "127.13.157.188" port: 33993 } }
I20260925 02:47:30.752404 3952 master_runner.cc:186] Error getting master registration for 127.13.157.189:42211: OK, instance_id { permanent_uuid: "97953dc98dfc46ff9fcfb0bd3e82281b" instance_seqno: 1790304450462868 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260925 02:47:30.756162 3952 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260925 02:47:30.756279 4069 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "55a954a30ce64942959f12b90c70cae4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "97953dc98dfc46ff9fcfb0bd3e82281b"
I20260925 02:47:30.756528 4069 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 97953dc98dfc46ff9fcfb0bd3e82281b [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:47:30.757946 4069 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 97953dc98dfc46ff9fcfb0bd3e82281b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 55a954a30ce64942959f12b90c70cae4 in term 1.
I20260925 02:47:30.758701 4193 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55a954a30ce64942959f12b90c70cae4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 55a954a30ce64942959f12b90c70cae4. Latest consensus state: current_term: 1 leader_uuid: "55a954a30ce64942959f12b90c70cae4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55a954a30ce64942959f12b90c70cae4" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 43651 } } peers { permanent_uuid: "97953dc98dfc46ff9fcfb0bd3e82281b" member_type: VOTER last_known_addr { host: "127.13.157.189" port: 42211 } } peers { permanent_uuid: "a48e29845e1e45ea8488399b9424a611" member_type: VOTER last_known_addr { host: "127.13.157.188" port: 33993 } } }
I20260925 02:47:30.758940 4193 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55a954a30ce64942959f12b90c70cae4 [sys.catalog]: This master's current role is: LEADER
I20260925 02:47:30.759130 4212 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97953dc98dfc46ff9fcfb0bd3e82281b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55a954a30ce64942959f12b90c70cae4" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 43651 } } peers { permanent_uuid: "97953dc98dfc46ff9fcfb0bd3e82281b" member_type: VOTER last_known_addr { host: "127.13.157.189" port: 42211 } } peers { permanent_uuid: "a48e29845e1e45ea8488399b9424a611" member_type: VOTER last_known_addr { host: "127.13.157.188" port: 33993 } } }
I20260925 02:47:30.759260 4212 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97953dc98dfc46ff9fcfb0bd3e82281b [sys.catalog]: This master's current role is: FOLLOWER
W20260925 02:47:30.766650 4206 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 55a954a30ce64942959f12b90c70cae4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 02:47:30.766738 4206 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 02:47:30.768131 4022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:30.753835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51930 (local address 127.13.157.190:43651)
0925 02:47:30.753943 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:30.753948 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:30.759601 (+ 5653us) server_negotiation.cc:407] Connection header received
0925 02:47:30.759777 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:30.759780 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:30.759825 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:30.759872 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:47:30.760751 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:30.761431 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:30.762507 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:30.762842 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:30.762944 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:30.764514 (+ 1570us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:47:30.764518 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:47:30.764520 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:47:30.764547 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:47:30.766129 (+ 1582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:47:30.766591 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:47:30.766595 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:47:30.766597 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:47:30.766682 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:47:30.767115 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:47:30.767118 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:47:30.767120 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:47:30.767338 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:47:30.767442 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:30.767813 (+ 371us) server_negotiation.cc:299] Negotiation successful
0925 02:47:30.767968 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260925 02:47:30.770475 4101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:30.753080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36698 (local address 127.13.157.189:42211)
0925 02:47:30.753208 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:30.753212 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:30.757002 (+ 3790us) server_negotiation.cc:407] Connection header received
0925 02:47:30.758365 (+ 1363us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:30.758368 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:30.758427 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:30.758496 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:47:30.759548 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:30.760242 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:30.761367 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:30.761682 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:30.761755 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:30.766924 (+ 5169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:47:30.766929 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:47:30.766932 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:47:30.766960 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:47:30.768629 (+ 1669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:47:30.769108 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:47:30.769111 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:47:30.769113 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:47:30.769199 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:47:30.769516 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:47:30.769520 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:47:30.769522 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:47:30.769738 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:47:30.769848 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:30.770210 (+ 362us) server_negotiation.cc:299] Negotiation successful
0925 02:47:30.770327 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
W20260925 02:47:30.771920 4226 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 97953dc98dfc46ff9fcfb0bd3e82281b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 02:47:30.772012 4226 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 02:47:30.774858 4178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:30.754452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33020 (local address 127.13.157.188:33993)
0925 02:47:30.754586 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:30.754590 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:30.760963 (+ 6373us) server_negotiation.cc:407] Connection header received
0925 02:47:30.761125 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:30.761128 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:30.761175 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:30.761221 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:47:30.763168 (+ 1947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:30.763920 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:30.764965 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:30.765321 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:30.765397 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:30.771077 (+ 5680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:47:30.771082 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:47:30.771084 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:47:30.771115 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:47:30.772829 (+ 1714us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:47:30.773352 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:47:30.773356 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:47:30.773358 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:47:30.773456 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:47:30.773778 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:47:30.773781 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:47:30.773783 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:47:30.774012 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:47:30.774148 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:30.774546 (+ 398us) server_negotiation.cc:299] Negotiation successful
0925 02:47:30.774692 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260925 02:47:30.775489 4228 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:47:30.780038 4228 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:47:30.788384 4146 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a48e29845e1e45ea8488399b9424a611 [term 1 FOLLOWER]: Refusing update from remote peer 55a954a30ce64942959f12b90c70cae4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:47:30.788506 4069 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 97953dc98dfc46ff9fcfb0bd3e82281b [term 1 FOLLOWER]: Refusing update from remote peer 55a954a30ce64942959f12b90c70cae4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:47:30.790050 4207 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 55a954a30ce64942959f12b90c70cae4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97953dc98dfc46ff9fcfb0bd3e82281b" member_type: VOTER last_known_addr { host: "127.13.157.189" port: 42211 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:47:30.790411 4193 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 55a954a30ce64942959f12b90c70cae4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a48e29845e1e45ea8488399b9424a611" member_type: VOTER last_known_addr { host: "127.13.157.188" port: 33993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:47:30.794842 4212 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97953dc98dfc46ff9fcfb0bd3e82281b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 55a954a30ce64942959f12b90c70cae4. Latest consensus state: current_term: 1 leader_uuid: "55a954a30ce64942959f12b90c70cae4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55a954a30ce64942959f12b90c70cae4" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 43651 } } peers { permanent_uuid: "97953dc98dfc46ff9fcfb0bd3e82281b" member_type: VOTER last_known_addr { host: "127.13.157.189" port: 42211 } } peers { permanent_uuid: "a48e29845e1e45ea8488399b9424a611" member_type: VOTER last_known_addr { host: "127.13.157.188" port: 33993 } } }
I20260925 02:47:30.794844 4181 sys_catalog.cc:455] T 00000000000000000000000000000000 P a48e29845e1e45ea8488399b9424a611 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 55a954a30ce64942959f12b90c70cae4. Latest consensus state: current_term: 1 leader_uuid: "55a954a30ce64942959f12b90c70cae4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55a954a30ce64942959f12b90c70cae4" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 43651 } } peers { permanent_uuid: "97953dc98dfc46ff9fcfb0bd3e82281b" member_type: VOTER last_known_addr { host: "127.13.157.189" port: 42211 } } peers { permanent_uuid: "a48e29845e1e45ea8488399b9424a611" member_type: VOTER last_known_addr { host: "127.13.157.188" port: 33993 } } }
I20260925 02:47:30.794991 4212 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97953dc98dfc46ff9fcfb0bd3e82281b [sys.catalog]: This master's current role is: FOLLOWER
I20260925 02:47:30.794996 4181 sys_catalog.cc:458] T 00000000000000000000000000000000 P a48e29845e1e45ea8488399b9424a611 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 02:47:30.797703 4193 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55a954a30ce64942959f12b90c70cae4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "55a954a30ce64942959f12b90c70cae4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55a954a30ce64942959f12b90c70cae4" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 43651 } } peers { permanent_uuid: "97953dc98dfc46ff9fcfb0bd3e82281b" member_type: VOTER last_known_addr { host: "127.13.157.189" port: 42211 } } peers { permanent_uuid: "a48e29845e1e45ea8488399b9424a611" member_type: VOTER last_known_addr { host: "127.13.157.188" port: 33993 } } }
I20260925 02:47:30.797837 4193 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55a954a30ce64942959f12b90c70cae4 [sys.catalog]: This master's current role is: LEADER
I20260925 02:47:30.798954 4181 sys_catalog.cc:455] T 00000000000000000000000000000000 P a48e29845e1e45ea8488399b9424a611 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "55a954a30ce64942959f12b90c70cae4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55a954a30ce64942959f12b90c70cae4" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 43651 } } peers { permanent_uuid: "97953dc98dfc46ff9fcfb0bd3e82281b" member_type: VOTER last_known_addr { host: "127.13.157.189" port: 42211 } } peers { permanent_uuid: "a48e29845e1e45ea8488399b9424a611" member_type: VOTER last_known_addr { host: "127.13.157.188" port: 33993 } } }
I20260925 02:47:30.799065 4181 sys_catalog.cc:458] T 00000000000000000000000000000000 P a48e29845e1e45ea8488399b9424a611 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 02:47:30.799039 4231 mvcc.cc:205] Tried to move back new op lower bound from 7333087030419808256 to 7333087030301630464. Current Snapshot: MvccSnapshot[applied={T|T < 7333087030419808256}]
I20260925 02:47:30.801306 4207 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55a954a30ce64942959f12b90c70cae4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "55a954a30ce64942959f12b90c70cae4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55a954a30ce64942959f12b90c70cae4" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 43651 } } peers { permanent_uuid: "97953dc98dfc46ff9fcfb0bd3e82281b" member_type: VOTER last_known_addr { host: "127.13.157.189" port: 42211 } } peers { permanent_uuid: "a48e29845e1e45ea8488399b9424a611" member_type: VOTER last_known_addr { host: "127.13.157.188" port: 33993 } } }
I20260925 02:47:30.801420 4207 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55a954a30ce64942959f12b90c70cae4 [sys.catalog]: This master's current role is: LEADER
I20260925 02:47:30.802181 4212 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97953dc98dfc46ff9fcfb0bd3e82281b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "55a954a30ce64942959f12b90c70cae4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55a954a30ce64942959f12b90c70cae4" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 43651 } } peers { permanent_uuid: "97953dc98dfc46ff9fcfb0bd3e82281b" member_type: VOTER last_known_addr { host: "127.13.157.189" port: 42211 } } peers { permanent_uuid: "a48e29845e1e45ea8488399b9424a611" member_type: VOTER last_known_addr { host: "127.13.157.188" port: 33993 } } }
I20260925 02:47:30.802294 4212 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97953dc98dfc46ff9fcfb0bd3e82281b [sys.catalog]: This master's current role is: FOLLOWER
I20260925 02:47:30.803628 4232 mvcc.cc:205] Tried to move back new op lower bound from 7333087030419808256 to 7333087030301630464. Current Snapshot: MvccSnapshot[applied={T|T < 7333087030419808256}]
I20260925 02:47:30.810359 4233 mvcc.cc:205] Tried to move back new op lower bound from 7333087030419808256 to 7333087030301630464. Current Snapshot: MvccSnapshot[applied={T|T < 7333087030419808256}]
I20260925 02:47:30.810959 4228 catalog_manager.cc:1347] Generated new cluster ID: 800ae578a3034d44a3da3fba3af39bef
I20260925 02:47:30.811033 4228 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 02:47:30.846809 4228 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 02:47:30.848184 4228 catalog_manager.cc:1504] Loading token signing keys...
I20260925 02:47:30.872913 4228 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 55a954a30ce64942959f12b90c70cae4: Generated new TSK 0
I20260925 02:47:30.873847 4228 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260925 02:47:30.918406 4022 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:47:30.915166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51940 (local address 127.13.157.190:43651)
0925 02:47:30.915299 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:30.915304 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:30.917296 (+ 1992us) server_negotiation.cc:407] Connection header received
0925 02:47:30.917500 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:30.917504 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:30.917557 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:30.917610 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:47:30.918265 (+ 655us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51940: BlockingRecv error: recv got EOF from 127.0.0.1:51940 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
W20260925 02:47:30.920267 4101 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:47:30.915648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36706 (local address 127.13.157.189:42211)
0925 02:47:30.915766 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:30.915770 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:30.918802 (+ 3032us) server_negotiation.cc:407] Connection header received
0925 02:47:30.919395 (+ 593us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:30.919399 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:30.919451 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:30.919506 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:47:30.920145 (+ 639us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36706: BlockingRecv error: recv got EOF from 127.0.0.1:36706 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260925 02:47:30.921240 4178 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:47:30.916467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33036 (local address 127.13.157.188:33993)
0925 02:47:30.916580 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:30.916585 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:30.919208 (+ 2623us) server_negotiation.cc:407] Connection header received
0925 02:47:30.920511 (+ 1303us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:30.920515 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:30.920568 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:30.920619 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:47:30.921117 (+ 498us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33036: BlockingRecv error: recv got EOF from 127.0.0.1:33036 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260925 02:47:30.926867 4101 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:47:30.925342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36716 (local address 127.13.157.189:42211)
0925 02:47:30.925449 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:30.925453 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:30.925886 (+ 433us) server_negotiation.cc:407] Connection header received
0925 02:47:30.926122 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:30.926125 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:30.926203 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:30.926257 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:47:30.926762 (+ 505us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36716: BlockingRecv error: recv got EOF from 127.0.0.1:36716 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
W20260925 02:47:30.932195 4101 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:47:30.930508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36718 (local address 127.13.157.189:42211)
0925 02:47:30.930620 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:30.930624 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:30.931235 (+ 611us) server_negotiation.cc:407] Connection header received
0925 02:47:30.931430 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:30.931433 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:30.931488 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:30.931542 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:47:30.932084 (+ 542us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36718: BlockingRecv error: recv got EOF from 127.0.0.1:36718 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260925 02:47:30.938257 4101 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:47:30.936571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36734 (local address 127.13.157.189:42211)
0925 02:47:30.936680 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:30.936684 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:30.937257 (+ 573us) server_negotiation.cc:407] Connection header received
0925 02:47:30.937470 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:30.937473 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:30.937532 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:30.937588 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:47:30.938144 (+ 556us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36734: BlockingRecv error: recv got EOF from 127.0.0.1:36734 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260925 02:47:30.946566 4101 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:47:30.944738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36742 (local address 127.13.157.189:42211)
0925 02:47:30.944881 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:30.944886 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:30.945542 (+ 656us) server_negotiation.cc:407] Connection header received
0925 02:47:30.945761 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:30.945766 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:30.945823 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:30.945878 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:47:30.946450 (+ 572us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36742: BlockingRecv error: recv got EOF from 127.0.0.1:36742 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
W20260925 02:47:30.958782 4101 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:47:30.957082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36754 (local address 127.13.157.189:42211)
0925 02:47:30.957201 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:30.957205 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:30.957795 (+ 590us) server_negotiation.cc:407] Connection header received
0925 02:47:30.958001 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:30.958005 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:30.958057 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:30.958108 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:47:30.958659 (+ 551us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36754: BlockingRecv error: recv got EOF from 127.0.0.1:36754 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260925 02:47:30.979156 4101 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:47:30.977990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36766 (local address 127.13.157.189:42211)
0925 02:47:30.978105 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:30.978109 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:30.978312 (+ 203us) server_negotiation.cc:407] Connection header received
0925 02:47:30.978454 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:30.978457 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:30.978506 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:30.978572 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:47:30.979048 (+ 476us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36766: BlockingRecv error: recv got EOF from 127.0.0.1:36766 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
W20260925 02:47:31.014448 4101 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:47:31.013346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36774 (local address 127.13.157.189:42211)
0925 02:47:31.013505 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:31.013509 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:31.013645 (+ 136us) server_negotiation.cc:407] Connection header received
0925 02:47:31.013738 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:31.013740 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:31.013795 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:31.013895 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:47:31.014289 (+ 394us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36774: BlockingRecv error: recv got EOF from 127.0.0.1:36774 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
W20260925 02:47:31.081887 4101 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:47:31.080711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36788 (local address 127.13.157.189:42211)
0925 02:47:31.080852 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:31.080856 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:31.081004 (+ 148us) server_negotiation.cc:407] Connection header received
0925 02:47:31.081155 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:31.081158 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:31.081208 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:31.081279 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:47:31.081741 (+ 462us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36788: BlockingRecv error: recv got EOF from 127.0.0.1:36788 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260925 02:47:31.213145 4101 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:47:31.211980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36794 (local address 127.13.157.189:42211)
0925 02:47:31.212138 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:31.212143 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:31.212282 (+ 139us) server_negotiation.cc:407] Connection header received
0925 02:47:31.212438 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:31.212442 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:31.212496 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:31.212567 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:47:31.213020 (+ 453us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36794: BlockingRecv error: recv got EOF from 127.0.0.1:36794 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260925 02:47:31.472600 4101 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:47:31.471421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36808 (local address 127.13.157.189:42211)
0925 02:47:31.471570 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:31.471574 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:31.471739 (+ 165us) server_negotiation.cc:407] Connection header received
0925 02:47:31.471907 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:31.471910 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:31.471958 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:31.472020 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:47:31.472456 (+ 436us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36808: BlockingRecv error: recv got EOF from 127.0.0.1:36808 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
I20260925 02:47:31.745852 4192 catalog_manager.cc:1259] Loaded cluster ID: 800ae578a3034d44a3da3fba3af39bef
I20260925 02:47:31.745959 4192 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a48e29845e1e45ea8488399b9424a611: loading cluster ID for follower catalog manager: success
I20260925 02:47:31.747664 4192 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a48e29845e1e45ea8488399b9424a611: acquiring CA information for follower catalog manager: success
I20260925 02:47:31.748821 4192 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a48e29845e1e45ea8488399b9424a611: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 02:47:31.773321 4226 catalog_manager.cc:1259] Loaded cluster ID: 800ae578a3034d44a3da3fba3af39bef
I20260925 02:47:31.773413 4226 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 97953dc98dfc46ff9fcfb0bd3e82281b: loading cluster ID for follower catalog manager: success
I20260925 02:47:31.775153 4226 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 97953dc98dfc46ff9fcfb0bd3e82281b: acquiring CA information for follower catalog manager: success
I20260925 02:47:31.776304 4226 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 97953dc98dfc46ff9fcfb0bd3e82281b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 02:47:31.990752 4304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:31.986865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60650 (local address 127.13.157.189:42211)
0925 02:47:31.987140 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:31.987144 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:31.987162 (+ 18us) server_negotiation.cc:407] Connection header received
0925 02:47:31.987284 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:31.987286 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:31.987442 (+ 156us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:31.987559 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 02:47:31.987799 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:31.988374 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:31.989288 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:31.989536 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:31.989627 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:31.989899 (+ 272us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 02:47:31.990334 (+ 435us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 02:47:31.990482 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:31.990550 (+ 68us) server_negotiation.cc:299] Negotiation successful
0925 02:47:31.990609 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":74,"threads_started":1}
I20260925 02:47:31.997653 4310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:31.993977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54790 (local address 127.13.157.188:33993)
0925 02:47:31.994206 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:31.994209 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:31.994221 (+ 12us) server_negotiation.cc:407] Connection header received
0925 02:47:31.994298 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:31.994301 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:31.994435 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:31.994544 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 02:47:31.994798 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:31.995378 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:31.996249 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:31.996487 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:31.996574 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:31.996829 (+ 255us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 02:47:31.997242 (+ 413us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 02:47:31.997334 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:31.997394 (+ 60us) server_negotiation.cc:299] Negotiation successful
0925 02:47:31.997486 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":88,"threads_started":1}
I20260925 02:47:32.004588 4304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:32.000679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60664 (local address 127.13.157.189:42211)
0925 02:47:32.000809 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:32.000812 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:32.001040 (+ 228us) server_negotiation.cc:407] Connection header received
0925 02:47:32.001230 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:32.001234 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:32.001287 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:32.001333 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 02:47:32.001685 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:32.002375 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:32.003584 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:32.003815 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:32.003920 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:32.004118 (+ 198us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 02:47:32.004221 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 02:47:32.004284 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:32.004393 (+ 109us) server_negotiation.cc:299] Negotiation successful
0925 02:47:32.004452 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260925 02:47:32.005028 4310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:32.001187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54802 (local address 127.13.157.188:33993)
0925 02:47:32.001293 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:32.001296 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:32.001309 (+ 13us) server_negotiation.cc:407] Connection header received
0925 02:47:32.001432 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:32.001435 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:32.001480 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:32.001538 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 02:47:32.001757 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:32.002381 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:32.003842 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:32.004114 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:32.004204 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:32.004601 (+ 397us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 02:47:32.004674 (+ 73us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 02:47:32.004717 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:32.004881 (+ 164us) server_negotiation.cc:299] Negotiation successful
0925 02:47:32.004932 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260925 02:47:32.005584 4317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:32.000723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42008 (local address 127.13.157.190:43651)
0925 02:47:32.000952 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:32.000957 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:32.001121 (+ 164us) server_negotiation.cc:407] Connection header received
0925 02:47:32.001222 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:32.001226 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:32.001392 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:32.001501 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 02:47:32.001953 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:32.002686 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:32.003887 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:32.004237 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:32.004322 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:32.004709 (+ 387us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 02:47:32.005048 (+ 339us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 02:47:32.005130 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:32.005389 (+ 259us) server_negotiation.cc:299] Negotiation successful
0925 02:47:32.005448 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":61,"threads_started":1}
I20260925 02:47:32.010035 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 3952
I20260925 02:47:32.017566 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 4027
I20260925 02:47:32.023458 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 4102
2026-09-25T02:47:32Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3040 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790304178325258-13942-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 02:47:32 dist-test-slave-kznw krb5kdc[4321](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 02:47:32 dist-test-slave-kznw krb5kdc[4321](info): set up 2 sockets
Sep 25 02:47:32 dist-test-slave-kznw krb5kdc[4321](info): commencing operation
krb5kdc: starting...
W20260925 02:47:32.982733 13942 mini_kdc.cc:121] Time spent starting KDC: real 0.938s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790304178325258-13942-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790304178325258-13942-0/krb5kdc/test-user.keytab.
Sep 25 02:47:33 dist-test-slave-kznw krb5kdc[4321](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304453, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T02:47:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T02:47:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.157.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.157.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.157.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab.
Entry for principal kudu/127.13.157.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab.
I20260925 02:47:33.106391 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790304178325258-13942-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790304178325258-13942-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790304178325258-13942-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.157.190:43961
--webserver_interface=127.13.157.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab
--principal=kudu/127.13.157.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:46025
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.157.190:43961,127.13.157.189:45923,127.13.157.188:36985
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:47:33.214171 4334 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260925 02:47:33.214464 4334 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:47:33.214500 4334 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:47:33.214563 4334 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:47:33.218535 4334 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:47:33.218593 4334 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:47:33.218619 4334 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:47:33.218639 4334 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:47:33.218667 4334 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:47:33.223925 4334 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:46025
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790304178325258-13942-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790304178325258-13942-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.157.190:43961,127.13.157.189:45923,127.13.157.188:36985
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.190.keytab
--principal=kudu/127.13.157.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.157.190:43961
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.157.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4334
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790304178325258-13942-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:47:33.224969 4334 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:47:33.225986 4334 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:47:33.234285 4342 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:47:33.234476 4334 server_base.cc:1034] running on GCE node
W20260925 02:47:33.234313 4339 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:47:33.234305 4340 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:47:33.235263 4334 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:47:33.236378 4334 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:47:33.237581 4334 hybrid_clock.cc:648] HybridClock initialized: now 1790304453237560 us; error 31 us; skew 500 ppm
Sep 25 02:47:33 dist-test-slave-kznw krb5kdc[4321](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304453, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:47:33.246017 4334 init.cc:377] Logged in from keytab as kudu/127.13.157.190@KRBTEST.COM (short username kudu)
I20260925 02:47:33.247074 4334 webserver.cc:460] Webserver started at http://127.13.157.190:45203/ using document root <none> and password file <none>
I20260925 02:47:33.247654 4334 fs_manager.cc:359] Metadata directory not provided
I20260925 02:47:33.247731 4334 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:47:33.247953 4334 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:47:33.250140 4334 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790304178325258-13942-0/minicluster-data/master-0/data/instance:
uuid: "8c95418adbd04368bfda33dd9bd3d64d"
format_stamp: "Formatted at 2026-09-25 02:47:33 on dist-test-slave-kznw"
server_key: "f21912ee8ca462c7b18b6050c5075758"
server_key_iv: "94a2d27b110c214c16c0a0c55fef7960"
server_key_version: "encryptionkey@0"
I20260925 02:47:33.250725 4334 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790304178325258-13942-0/minicluster-data/master-0/wal/instance:
uuid: "8c95418adbd04368bfda33dd9bd3d64d"
format_stamp: "Formatted at 2026-09-25 02:47:33 on dist-test-slave-kznw"
server_key: "f21912ee8ca462c7b18b6050c5075758"
server_key_iv: "94a2d27b110c214c16c0a0c55fef7960"
server_key_version: "encryptionkey@0"
I20260925 02:47:33.254619 4334 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.000s
I20260925 02:47:33.257258 4349 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:47:33.258637 4334 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260925 02:47:33.258780 4334 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790304178325258-13942-0/minicluster-data/master-0/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790304178325258-13942-0/minicluster-data/master-0/wal
uuid: "8c95418adbd04368bfda33dd9bd3d64d"
format_stamp: "Formatted at 2026-09-25 02:47:33 on dist-test-slave-kznw"
server_key: "f21912ee8ca462c7b18b6050c5075758"
server_key_iv: "94a2d27b110c214c16c0a0c55fef7960"
server_key_version: "encryptionkey@0"
I20260925 02:47:33.258936 4334 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790304178325258-13942-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790304178325258-13942-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790304178325258-13942-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:47:33.271533 4334 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:47:33.274626 4334 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:47:33.274806 4334 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:47:33.281654 4334 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.190:43961
I20260925 02:47:33.281940 4401 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.190:43961 every 8 connection(s)
I20260925 02:47:33.282636 4334 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790304178325258-13942-0/minicluster-data/master-0/data/info.pb
I20260925 02:47:33.285372 4402 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:47:33.289410 4402 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.157.190" port: 43961 } has no permanent_uuid. Determining permanent_uuid...
I20260925 02:47:33.292695 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 4334
I20260925 02:47:33.292805 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790304178325258-13942-0/minicluster-data/master-0/wal/instance
I20260925 02:47:33.293085 13942 external_mini_cluster.cc:1428] Setting key d83338c4a68e48ed9ba14a7aef2d7d72
Sep 25 02:47:33 dist-test-slave-kznw krb5kdc[4321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304453, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.190@KRBTEST.COM for kudu/127.13.157.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.157.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.157.189@KRBTEST.COM" created.
I20260925 02:47:33.304136 4405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:33.291666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49776 (local address 127.13.157.190:43961)
0925 02:47:33.292113 (+ 447us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:33.292136 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:33.292284 (+ 148us) server_negotiation.cc:407] Connection header received
0925 02:47:33.293400 (+ 1116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:33.293405 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:33.293597 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:33.293769 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:47:33.294610 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:33.295531 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:33.296522 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:33.297113 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:33.297283 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:33.299505 (+ 2222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:47:33.299512 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:47:33.299526 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:47:33.299552 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:47:33.301119 (+ 1567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:47:33.301609 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:47:33.301614 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:47:33.301615 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:47:33.301716 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:47:33.302090 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:47:33.302098 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:47:33.302099 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:47:33.302507 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:47:33.302609 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:33.303058 (+ 449us) server_negotiation.cc:299] Negotiation successful
0925 02:47:33.303260 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":326,"spinlock_wait_cycles":50944,"thread_start_us":158,"threads_started":1}
I20260925 02:47:33.304141 4404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:33.291665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:43961 (local address 127.0.0.1:49776)
0925 02:47:33.292149 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0925 02:47:33.292188 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0925 02:47:33.293083 (+ 895us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:47:33.293773 (+ 690us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:47:33.293779 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:47:33.294219 (+ 440us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:47:33.294463 (+ 244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:47:33.294477 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:47:33.295645 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:47:33.295647 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:47:33.296386 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:47:33.296397 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:47:33.297273 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:47:33.297276 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:47:33.297376 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:33.297902 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:47:33.297919 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:47:33.299346 (+ 1427us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:47:33.301218 (+ 1872us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:47:33.301226 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:47:33.301234 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:47:33.301521 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:47:33.301831 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:47:33.301836 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:47:33.301838 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:47:33.301973 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:47:33.302627 (+ 654us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:47:33.302633 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:47:33.302765 (+ 132us) client_negotiation.cc:769] Sending connection context
0925 02:47:33.302967 (+ 202us) client_negotiation.cc:240] Negotiation successful
0925 02:47:33.303433 (+ 466us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"mutex_wait_us":98,"thread_start_us":138,"threads_started":1}
I20260925 02:47:33.306499 4402 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.157.189" port: 45923 } has no permanent_uuid. Determining permanent_uuid...
I20260925 02:47:33.307233 4404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:33.306998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.189:45923 (local address 127.0.0.1:58856)
0925 02:47:33.307108 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0925 02:47:33.307177 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.189:45923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260925 02:47:33.307431 4351 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.157.189:45923: connect: Connection refused (error 111)
I20260925 02:47:33.308180 4404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:33.307991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.189:45923 (local address 127.0.0.1:58862)
0925 02:47:33.308093 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0925 02:47:33.308133 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.189:45923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260925 02:47:33.308300 4402 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.157.189:45923: Network error: Client connection negotiation failed: client connection to 127.13.157.189:45923: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.157.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.189.keytab.
Entry for principal kudu/127.13.157.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.189.keytab.
I20260925 02:47:33.318176 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790304178325258-13942-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790304178325258-13942-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790304178325258-13942-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790304178325258-13942-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.157.189:45923
--webserver_interface=127.13.157.189
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.189.keytab
--principal=kudu/127.13.157.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:46025
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.157.190:43961,127.13.157.189:45923,127.13.157.188:36985
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 02:47:33.357587 4402 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.157.189" port: 45923 } attempt: 1
I20260925 02:47:33.358541 4404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:33.358188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.189:45923 (local address 127.0.0.1:58864)
0925 02:47:33.358383 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0925 02:47:33.358454 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.189:45923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260925 02:47:33.359303 4404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:33.359089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.189:45923 (local address 127.0.0.1:58872)
0925 02:47:33.359196 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0925 02:47:33.359243 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.189:45923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260925 02:47:33.359515 4402 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.157.189:45923: Network error: Client connection negotiation failed: client connection to 127.13.157.189:45923: connect: Connection refused (error 111)
I20260925 02:47:33.427686 4402 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.157.189" port: 45923 } attempt: 2
I20260925 02:47:33.428602 4404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:33.428240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.189:45923 (local address 127.0.0.1:58888)
0925 02:47:33.428425 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0925 02:47:33.428494 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.189:45923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
W20260925 02:47:33.428704 4409 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260925 02:47:33.428996 4409 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:47:33.429039 4409 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:47:33.429102 4409 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260925 02:47:33.429373 4404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:33.429133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.189:45923 (local address 127.0.0.1:58900)
0925 02:47:33.429252 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0925 02:47:33.429299 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.189:45923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260925 02:47:33.429491 4402 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.157.189:45923: Network error: Client connection negotiation failed: client connection to 127.13.157.189:45923: connect: Connection refused (error 111)
W20260925 02:47:33.433197 4409 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:47:33.433250 4409 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:47:33.433276 4409 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:47:33.433308 4409 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:47:33.433337 4409 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:47:33.438551 4409 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:46025
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790304178325258-13942-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790304178325258-13942-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.13.157.190:43961,127.13.157.189:45923,127.13.157.188:36985
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.189.keytab
--principal=kudu/127.13.157.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.157.189:45923
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790304178325258-13942-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.13.157.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4409
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790304178325258-13942-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:47:33.439653 4409 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:47:33.440676 4409 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:47:33.449163 4417 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:47:33.449163 4415 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:47:33.449177 4414 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:47:33.449430 4409 server_base.cc:1034] running on GCE node
I20260925 02:47:33.450096 4409 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:47:33.451257 4409 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:47:33.452463 4409 hybrid_clock.cc:648] HybridClock initialized: now 1790304453452443 us; error 36 us; skew 500 ppm
Sep 25 02:47:33 dist-test-slave-kznw krb5kdc[4321](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304453, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:47:33.455199 4409 init.cc:377] Logged in from keytab as kudu/127.13.157.189@KRBTEST.COM (short username kudu)
I20260925 02:47:33.456126 4409 webserver.cc:460] Webserver started at http://127.13.157.189:37693/ using document root <none> and password file <none>
I20260925 02:47:33.456662 4409 fs_manager.cc:359] Metadata directory not provided
I20260925 02:47:33.456727 4409 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:47:33.456915 4409 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:47:33.459007 4409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790304178325258-13942-0/minicluster-data/master-1/data/instance:
uuid: "f120b693139f461d98983032354be924"
format_stamp: "Formatted at 2026-09-25 02:47:33 on dist-test-slave-kznw"
server_key: "0befff214ac346197bd7c3c3aa976406"
server_key_iv: "dbb5153a074092c033071676f7faf784"
server_key_version: "encryptionkey@0"
I20260925 02:47:33.459564 4409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790304178325258-13942-0/minicluster-data/master-1/wal/instance:
uuid: "f120b693139f461d98983032354be924"
format_stamp: "Formatted at 2026-09-25 02:47:33 on dist-test-slave-kznw"
server_key: "0befff214ac346197bd7c3c3aa976406"
server_key_iv: "dbb5153a074092c033071676f7faf784"
server_key_version: "encryptionkey@0"
I20260925 02:47:33.463268 4409 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260925 02:47:33.465646 4424 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:47:33.466881 4409 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260925 02:47:33.467034 4409 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790304178325258-13942-0/minicluster-data/master-1/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790304178325258-13942-0/minicluster-data/master-1/wal
uuid: "f120b693139f461d98983032354be924"
format_stamp: "Formatted at 2026-09-25 02:47:33 on dist-test-slave-kznw"
server_key: "0befff214ac346197bd7c3c3aa976406"
server_key_iv: "dbb5153a074092c033071676f7faf784"
server_key_version: "encryptionkey@0"
I20260925 02:47:33.467133 4409 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790304178325258-13942-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790304178325258-13942-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790304178325258-13942-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:47:33.487274 4409 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:47:33.490244 4409 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:47:33.490427 4409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:47:33.497224 4409 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.189:45923
I20260925 02:47:33.497504 4476 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.189:45923 every 8 connection(s)
I20260925 02:47:33.498270 4409 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790304178325258-13942-0/minicluster-data/master-1/data/info.pb
I20260925 02:47:33.501068 4477 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:47:33.504307 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 4409
I20260925 02:47:33.504436 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790304178325258-13942-0/minicluster-data/master-1/wal/instance
I20260925 02:47:33.504784 13942 external_mini_cluster.cc:1428] Setting key 21c5d50b60e96c3351fde9e980bd4e2c
I20260925 02:47:33.505142 4477 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.157.190" port: 43961 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.13.157.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.157.188@KRBTEST.COM" created.
Sep 25 02:47:33 dist-test-slave-kznw krb5kdc[4321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304453, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.189@KRBTEST.COM for kudu/127.13.157.190@KRBTEST.COM
I20260925 02:47:33.519395 4405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:33.507299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49782 (local address 127.13.157.190:43961)
0925 02:47:33.507517 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:33.507522 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:33.508332 (+ 810us) server_negotiation.cc:407] Connection header received
0925 02:47:33.509305 (+ 973us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:33.509310 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:33.509375 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:33.509500 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:47:33.510649 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:33.511483 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:33.512648 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:33.512884 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:33.512974 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:33.515997 (+ 3023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:47:33.516000 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:47:33.516002 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:47:33.516029 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:47:33.517342 (+ 1313us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:47:33.517826 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:47:33.517829 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:47:33.517831 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:47:33.517906 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:47:33.518220 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:47:33.518224 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:47:33.518225 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:47:33.518445 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:47:33.518536 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:33.519152 (+ 616us) server_negotiation.cc:299] Negotiation successful
0925 02:47:33.519266 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260925 02:47:33.520511 4480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:33.507667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:43961 (local address 127.0.0.1:49782)
0925 02:47:33.508157 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0925 02:47:33.508210 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0925 02:47:33.509094 (+ 884us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:47:33.509665 (+ 571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:47:33.509672 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:47:33.510034 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:47:33.510495 (+ 461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:47:33.510507 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:47:33.511637 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:47:33.511640 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:47:33.512526 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:47:33.512535 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:47:33.513006 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:47:33.513008 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:47:33.513159 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:33.513706 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:47:33.513722 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:47:33.515822 (+ 2100us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:47:33.517463 (+ 1641us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:47:33.517469 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:47:33.517485 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:47:33.517732 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:47:33.517996 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:47:33.517999 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:47:33.518001 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:47:33.518133 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:47:33.518547 (+ 414us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:47:33.518554 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:47:33.518828 (+ 274us) client_negotiation.cc:769] Sending connection context
0925 02:47:33.519056 (+ 228us) client_negotiation.cc:240] Negotiation successful
0925 02:47:33.519416 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":138,"threads_started":1}
I20260925 02:47:33.520638 4402 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.157.189" port: 45923 } attempt: 3
I20260925 02:47:33.522253 4477 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.157.189" port: 45923 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 02:47:33 dist-test-slave-kznw krb5kdc[4321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304453, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.190@KRBTEST.COM for kudu/127.13.157.189@KRBTEST.COM
Sep 25 02:47:33 dist-test-slave-kznw krb5kdc[4321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304453, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.189@KRBTEST.COM for kudu/127.13.157.189@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.157.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.188.keytab.
Entry for principal kudu/127.13.157.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.188.keytab.
I20260925 02:47:33.529982 13942 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
/tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790304178325258-13942-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790304178325258-13942-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790304178325258-13942-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790304178325258-13942-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.157.188:36985
--webserver_interface=127.13.157.188
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.188.keytab
--principal=kudu/127.13.157.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.157.148:46025
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.157.190:43961,127.13.157.189:45923,127.13.157.188:36985
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790304178325258-13942-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790304178325258-13942-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790304178325258-13942-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 02:47:33.531675 4480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:33.522762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.189:45923 (local address 127.0.0.1:58916)
0925 02:47:33.522871 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0925 02:47:33.522884 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 02:47:33.522980 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:47:33.523343 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:47:33.523346 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:47:33.523506 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:47:33.523618 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:47:33.523625 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:47:33.524351 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:47:33.524354 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:47:33.524996 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:47:33.525006 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:47:33.525499 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:47:33.525502 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:47:33.525585 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:33.525928 (+ 343us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:47:33.525941 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:47:33.527712 (+ 1771us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:47:33.530013 (+ 2301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:47:33.530016 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:47:33.530018 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:47:33.530247 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:47:33.530564 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:47:33.530567 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:47:33.530569 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:47:33.530655 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:47:33.531234 (+ 579us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:47:33.531236 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:47:33.531306 (+ 70us) client_negotiation.cc:769] Sending connection context
0925 02:47:33.531393 (+ 87us) client_negotiation.cc:240] Negotiation successful
0925 02:47:33.531512 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260925 02:47:33.531675 4404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:33.521044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.189:45923 (local address 127.0.0.1:58914)
0925 02:47:33.521373 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0925 02:47:33.521388 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 02:47:33.521466 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:47:33.521979 (+ 513us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:47:33.521982 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:47:33.522155 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:47:33.522681 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:47:33.522692 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:47:33.523986 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:47:33.523990 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:47:33.524673 (+ 683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:47:33.524685 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:47:33.525337 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:47:33.525340 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:47:33.525445 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:33.525832 (+ 387us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:47:33.525844 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:47:33.527157 (+ 1313us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:47:33.530038 (+ 2881us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:47:33.530041 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:47:33.530044 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:47:33.530283 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:47:33.530615 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:47:33.530618 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:47:33.530620 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:47:33.530703 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:47:33.531198 (+ 495us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:47:33.531201 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:47:33.531297 (+ 96us) client_negotiation.cc:769] Sending connection context
0925 02:47:33.531378 (+ 81us) client_negotiation.cc:240] Negotiation successful
0925 02:47:33.531512 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258}
I20260925 02:47:33.532321 4483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:33.522809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58916 (local address 127.13.157.189:45923)
0925 02:47:33.523145 (+ 336us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:33.523151 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:33.523162 (+ 11us) server_negotiation.cc:407] Connection header received
0925 02:47:33.523190 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:33.523192 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:33.523232 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:33.523363 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:47:33.523712 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:33.524254 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:33.525126 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:33.525405 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:33.525480 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:33.527839 (+ 2359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:47:33.527843 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:47:33.527846 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:47:33.527872 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:47:33.529895 (+ 2023us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:47:33.530360 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:47:33.530366 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:47:33.530368 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:47:33.530469 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:47:33.530741 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:47:33.530744 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:47:33.530746 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:47:33.531075 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:47:33.531200 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:33.531978 (+ 778us) server_negotiation.cc:299] Negotiation successful
0925 02:47:33.532138 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":7,"server-negotiator.queue_time_us":283,"thread_start_us":158,"threads_started":1}
I20260925 02:47:33.532339 4482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:33.521184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58914 (local address 127.13.157.189:45923)
0925 02:47:33.521568 (+ 384us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:33.521580 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:33.521603 (+ 23us) server_negotiation.cc:407] Connection header received
0925 02:47:33.521644 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:33.521649 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:33.521855 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:33.521978 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:47:33.522822 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:33.523858 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:33.524787 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:33.525242 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:33.525335 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:33.527277 (+ 1942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:47:33.527287 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:47:33.527302 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:47:33.527323 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:47:33.529917 (+ 2594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:47:33.530382 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:47:33.530385 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:47:33.530386 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:47:33.530512 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:47:33.530801 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:47:33.530804 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:47:33.530806 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:47:33.531075 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:47:33.531946 (+ 871us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:33.532006 (+ 60us) server_negotiation.cc:299] Negotiation successful
0925 02:47:33.532189 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":99,"server-negotiator.queue_time_us":291,"spinlock_wait_cycles":768,"thread_start_us":94,"threads_started":1}
I20260925 02:47:33.534032 4402 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.157.188" port: 36985 } has no permanent_uuid. Determining permanent_uuid...
I20260925 02:47:33.534209 4477 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.157.188" port: 36985 } has no permanent_uuid. Determining permanent_uuid...
I20260925 02:47:33.534816 4480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:33.534593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.188:36985 (local address 127.0.0.1:38472)
0925 02:47:33.534698 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0925 02:47:33.534766 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.188:36985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260925 02:47:33.534977 4404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:33.534646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.188:36985 (local address 127.0.0.1:38484)
0925 02:47:33.534823 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0925 02:47:33.534873 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.188:36985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112}
W20260925 02:47:33.534991 4426 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.157.188:36985: connect: Connection refused (error 111)
I20260925 02:47:33.535753 4404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:33.535454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.188:36985 (local address 127.0.0.1:38492)
0925 02:47:33.535633 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0925 02:47:33.535682 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.188:36985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116}
I20260925 02:47:33.535869 4480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:33.535628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.188:36985 (local address 127.0.0.1:38494)
0925 02:47:33.535752 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0925 02:47:33.535809 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.188:36985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260925 02:47:33.535929 4402 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.157.188:36985: Network error: Client connection negotiation failed: client connection to 127.13.157.188:36985: connect: Connection refused (error 111)
W20260925 02:47:33.536048 4477 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.157.188:36985: Network error: Client connection negotiation failed: client connection to 127.13.157.188:36985: connect: Connection refused (error 111)
I20260925 02:47:33.567082 4402 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.157.188" port: 36985 } attempt: 1
I20260925 02:47:33.567943 4404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:33.567607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.188:36985 (local address 127.0.0.1:38504)
0925 02:47:33.567777 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0925 02:47:33.567849 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.188:36985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260925 02:47:33.568670 4404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:33.568459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.188:36985 (local address 127.0.0.1:38518)
0925 02:47:33.568566 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0925 02:47:33.568610 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.188:36985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260925 02:47:33.568779 4402 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.157.188:36985: Network error: Client connection negotiation failed: client connection to 127.13.157.188:36985: connect: Connection refused (error 111)
I20260925 02:47:33.585289 4477 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.157.188" port: 36985 } attempt: 1
I20260925 02:47:33.586169 4480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:33.585871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.188:36985 (local address 127.0.0.1:38522)
0925 02:47:33.586014 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0925 02:47:33.586080 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.188:36985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260925 02:47:33.586885 4480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:33.586660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.188:36985 (local address 127.0.0.1:38534)
0925 02:47:33.586770 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0925 02:47:33.586821 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.188:36985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260925 02:47:33.587073 4477 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.157.188:36985: Network error: Client connection negotiation failed: client connection to 127.13.157.188:36985: connect: Connection refused (error 111)
W20260925 02:47:33.638170 4484 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260925 02:47:33.638423 4484 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:47:33.638447 4484 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:47:33.638502 4484 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:47:33.642426 4484 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:47:33.642484 4484 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:47:33.642501 4484 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:47:33.642516 4484 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:47:33.642530 4484 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:47:33.643913 4402 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.157.188" port: 36985 } attempt: 2
I20260925 02:47:33.644685 4404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:33.644404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.188:36985 (local address 127.0.0.1:38548)
0925 02:47:33.644541 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0925 02:47:33.644604 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.188:36985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260925 02:47:33.645931 4404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:33.645718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.188:36985 (local address 127.0.0.1:38558)
0925 02:47:33.645821 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0925 02:47:33.645871 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.188:36985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260925 02:47:33.646080 4402 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.157.188:36985: Network error: Client connection negotiation failed: client connection to 127.13.157.188:36985: connect: Connection refused (error 111)
I20260925 02:47:33.648129 4484 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.157.148:46025
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790304178325258-13942-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790304178325258-13942-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.13.157.190:43961,127.13.157.189:45923,127.13.157.188:36985
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790304178325258-13942-0/krb5kdc/kudu_127.13.157.188.keytab
--principal=kudu/127.13.157.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.157.188:36985
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790304178325258-13942-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.13.157.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4484
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790304178325258-13942-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 02:09:23 UTC on ccycloud.quasar-wqmybt.root.comops.site
build id 4671760
I20260925 02:47:33.649127 4484 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:47:33.650102 4484 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 02:47:33.655212 4477 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.157.188" port: 36985 } attempt: 2
I20260925 02:47:33.655886 4480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:33.655638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.188:36985 (local address 127.0.0.1:38566)
0925 02:47:33.655759 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0925 02:47:33.655813 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.188:36985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260925 02:47:33.656563 4480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:33.656359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.188:36985 (local address 127.0.0.1:38568)
0925 02:47:33.656438 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0925 02:47:33.656481 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.157.188:36985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260925 02:47:33.656718 4477 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.157.188:36985: Network error: Client connection negotiation failed: client connection to 127.13.157.188:36985: connect: Connection refused (error 111)
W20260925 02:47:33.658452 4491 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:47:33.658463 4490 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:47:33.658521 4484 server_base.cc:1034] running on GCE node
W20260925 02:47:33.658458 4493 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:47:33.659313 4484 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:47:33.660390 4484 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:47:33.661567 4484 hybrid_clock.cc:648] HybridClock initialized: now 1790304453661555 us; error 23 us; skew 500 ppm
Sep 25 02:47:33 dist-test-slave-kznw krb5kdc[4321](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304453, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:47:33.664223 4484 init.cc:377] Logged in from keytab as kudu/127.13.157.188@KRBTEST.COM (short username kudu)
I20260925 02:47:33.665086 4484 webserver.cc:460] Webserver started at http://127.13.157.188:36737/ using document root <none> and password file <none>
I20260925 02:47:33.665589 4484 fs_manager.cc:359] Metadata directory not provided
I20260925 02:47:33.665650 4484 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:47:33.665817 4484 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:47:33.667832 4484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790304178325258-13942-0/minicluster-data/master-2/data/instance:
uuid: "1f0f0a05d42b43e88eb8ac891312b84d"
format_stamp: "Formatted at 2026-09-25 02:47:33 on dist-test-slave-kznw"
server_key: "9f73ed6c44430f081ea5da7cadc55856"
server_key_iv: "943e3601022d35a05714de445613cee9"
server_key_version: "encryptionkey@0"
I20260925 02:47:33.668349 4484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790304178325258-13942-0/minicluster-data/master-2/wal/instance:
uuid: "1f0f0a05d42b43e88eb8ac891312b84d"
format_stamp: "Formatted at 2026-09-25 02:47:33 on dist-test-slave-kznw"
server_key: "9f73ed6c44430f081ea5da7cadc55856"
server_key_iv: "943e3601022d35a05714de445613cee9"
server_key_version: "encryptionkey@0"
I20260925 02:47:33.672544 4484 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260925 02:47:33.675180 4500 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:47:33.676510 4484 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260925 02:47:33.676623 4484 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790304178325258-13942-0/minicluster-data/master-2/data,/tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790304178325258-13942-0/minicluster-data/master-2/wal
uuid: "1f0f0a05d42b43e88eb8ac891312b84d"
format_stamp: "Formatted at 2026-09-25 02:47:33 on dist-test-slave-kznw"
server_key: "9f73ed6c44430f081ea5da7cadc55856"
server_key_iv: "943e3601022d35a05714de445613cee9"
server_key_version: "encryptionkey@0"
I20260925 02:47:33.676743 4484 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790304178325258-13942-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790304178325258-13942-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790304178325258-13942-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:47:33.698969 4484 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:47:33.701953 4484 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:47:33.702121 4484 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:47:33.708626 4484 rpc_server.cc:307] RPC server started. Bound to: 127.13.157.188:36985
I20260925 02:47:33.708885 4552 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.157.188:36985 every 8 connection(s)
I20260925 02:47:33.709563 4484 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790304178325258-13942-0/minicluster-data/master-2/data/info.pb
I20260925 02:47:33.712234 4553 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:47:33.715791 4553 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.157.190" port: 43961 } has no permanent_uuid. Determining permanent_uuid...
I20260925 02:47:33.716367 13942 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu as pid 4484
I20260925 02:47:33.716501 13942 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksiQ9fF/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790304178325258-13942-0/minicluster-data/master-2/wal/instance
I20260925 02:47:33.716802 13942 external_mini_cluster.cc:1428] Setting key b559c7466e692522348ff05687ef727c
Sep 25 02:47:33 dist-test-slave-kznw krb5kdc[4321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304453, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.188@KRBTEST.COM for kudu/127.13.157.190@KRBTEST.COM
Sep 25 02:47:33 dist-test-slave-kznw krb5kdc[4321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304453, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.157.188@KRBTEST.COM
I20260925 02:47:33.728366 4405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:33.717732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49786 (local address 127.13.157.190:43961)
0925 02:47:33.717879 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:33.717884 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:33.718688 (+ 804us) server_negotiation.cc:407] Connection header received
0925 02:47:33.719569 (+ 881us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:33.719572 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:33.719625 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:33.719694 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:47:33.720599 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:33.721099 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:33.722127 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:33.722382 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:33.722458 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:33.725047 (+ 2589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:47:33.725050 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:47:33.725052 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:47:33.725078 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:47:33.726472 (+ 1394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:47:33.726977 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:47:33.726980 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:47:33.726982 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:47:33.727058 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:47:33.727368 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:47:33.727372 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:47:33.727373 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:47:33.727559 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:47:33.727642 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:33.728092 (+ 450us) server_negotiation.cc:299] Negotiation successful
0925 02:47:33.728206 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260925 02:47:33.729131 4556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:33.718017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.190:43961 (local address 127.0.0.1:49786)
0925 02:47:33.718535 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0925 02:47:33.718585 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0925 02:47:33.719354 (+ 769us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:47:33.719763 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:47:33.719770 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:47:33.720196 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:47:33.720458 (+ 262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:47:33.720471 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:47:33.721197 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:47:33.721200 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:47:33.722016 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:47:33.722026 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:47:33.722509 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:47:33.722513 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:47:33.722683 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:33.723302 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:47:33.723322 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:47:33.724903 (+ 1581us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:47:33.726569 (+ 1666us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:47:33.726577 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:47:33.726586 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:47:33.726867 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:47:33.727139 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:47:33.727142 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:47:33.727144 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:47:33.727274 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:47:33.727654 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:47:33.727660 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:47:33.727876 (+ 216us) client_negotiation.cc:769] Sending connection context
0925 02:47:33.728014 (+ 138us) client_negotiation.cc:240] Negotiation successful
0925 02:47:33.728284 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":137,"threads_started":1}
I20260925 02:47:33.729136 4557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:33.718017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38576 (local address 127.13.157.188:36985)
0925 02:47:33.718502 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:33.718519 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:33.718558 (+ 39us) server_negotiation.cc:407] Connection header received
0925 02:47:33.719352 (+ 794us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:33.719389 (+ 37us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:33.719628 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:33.719809 (+ 181us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:47:33.720451 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:33.721436 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:33.722264 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:33.722704 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:33.722802 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:33.725426 (+ 2624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:47:33.725431 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:47:33.725440 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:47:33.725460 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:47:33.726950 (+ 1490us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:47:33.727338 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:47:33.727344 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:47:33.727345 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:47:33.727423 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:47:33.727672 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:47:33.727675 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:47:33.727677 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:47:33.727910 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:47:33.728080 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:33.728301 (+ 221us) server_negotiation.cc:299] Negotiation successful
0925 02:47:33.728565 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":376,"spinlock_wait_cycles":41216,"thread_start_us":144,"threads_started":1}
I20260925 02:47:33.730280 4553 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.157.189" port: 45923 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 02:47:33 dist-test-slave-kznw krb5kdc[4321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304453, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.188@KRBTEST.COM for kudu/127.13.157.189@KRBTEST.COM
Sep 25 02:47:33 dist-test-slave-kznw krb5kdc[4321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304453, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.157.190@KRBTEST.COM
I20260925 02:47:33.737880 4556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:33.730704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.189:45923 (local address 127.0.0.1:58924)
0925 02:47:33.730819 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0925 02:47:33.730832 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 02:47:33.730927 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:47:33.731180 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:47:33.731182 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:47:33.731331 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:47:33.731452 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:47:33.731460 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:47:33.732292 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:47:33.732294 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:47:33.732808 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:47:33.732817 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:47:33.733258 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:47:33.733262 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:47:33.733357 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:33.733780 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:47:33.733791 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:47:33.735013 (+ 1222us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:47:33.736606 (+ 1593us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:47:33.736609 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:47:33.736611 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:47:33.736784 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:47:33.737023 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:47:33.737026 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:47:33.737028 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:47:33.737101 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:47:33.737441 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:47:33.737444 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:47:33.737530 (+ 86us) client_negotiation.cc:769] Sending connection context
0925 02:47:33.737612 (+ 82us) client_negotiation.cc:240] Negotiation successful
0925 02:47:33.737728 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260925 02:47:33.737939 4482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:33.730810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58924 (local address 127.13.157.189:45923)
0925 02:47:33.730980 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:33.730985 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:33.730998 (+ 13us) server_negotiation.cc:407] Connection header received
0925 02:47:33.731036 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:33.731038 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:33.731078 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:33.731144 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:47:33.731604 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:33.732181 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:33.732929 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:33.733166 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:33.733238 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:33.735133 (+ 1895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:47:33.735136 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:47:33.735137 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:47:33.735161 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:47:33.736502 (+ 1341us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:47:33.736865 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:47:33.736868 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:47:33.736870 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:47:33.736944 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:47:33.737176 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:47:33.737179 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:47:33.737180 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:47:33.737357 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:47:33.737446 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:33.737674 (+ 228us) server_negotiation.cc:299] Negotiation successful
0925 02:47:33.737779 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260925 02:47:33.738976 4405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:33.731373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49796 (local address 127.13.157.190:43961)
0925 02:47:33.731473 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:33.731476 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:33.731490 (+ 14us) server_negotiation.cc:407] Connection header received
0925 02:47:33.731621 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:33.731624 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:33.731667 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:33.731737 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:47:33.732227 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:33.732704 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:33.733553 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:33.733841 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:33.733917 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:33.736110 (+ 2193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:47:33.736113 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:47:33.736116 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:47:33.736139 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:47:33.737410 (+ 1271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:47:33.737775 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:47:33.737778 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:47:33.737780 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:47:33.737851 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:47:33.738114 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:47:33.738117 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:47:33.738119 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:47:33.738323 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:47:33.738542 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:33.738681 (+ 139us) server_negotiation.cc:299] Negotiation successful
0925 02:47:33.738783 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260925 02:47:33.738986 4553 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.157.188" port: 36985 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 02:47:33 dist-test-slave-kznw krb5kdc[4321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304453, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.188@KRBTEST.COM for kudu/127.13.157.188@KRBTEST.COM
Sep 25 02:47:33 dist-test-slave-kznw krb5kdc[4321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304453, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.157.189@KRBTEST.COM
I20260925 02:47:33.745245 4402 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.157.188" port: 36985 } attempt: 3
I20260925 02:47:33.747648 4557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:33.739488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38582 (local address 127.13.157.188:36985)
0925 02:47:33.739574 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:33.739578 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:33.739589 (+ 11us) server_negotiation.cc:407] Connection header received
0925 02:47:33.739658 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:33.739661 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:33.739692 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:33.739753 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:47:33.740091 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:33.740680 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:33.741653 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:33.741952 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:33.742038 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:33.744328 (+ 2290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:47:33.744332 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:47:33.744334 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:47:33.744359 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:47:33.745708 (+ 1349us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:47:33.746475 (+ 767us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:47:33.746477 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:47:33.746479 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:47:33.746553 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:47:33.746836 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:47:33.746839 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:47:33.746841 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:47:33.747059 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:47:33.747131 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:33.747400 (+ 269us) server_negotiation.cc:299] Negotiation successful
0925 02:47:33.747524 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":37,"server-negotiator.queue_time_us":41}
I20260925 02:47:33.747648 4556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:33.739425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.188:36985 (local address 127.0.0.1:38582)
0925 02:47:33.739515 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0925 02:47:33.739527 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 02:47:33.739594 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:47:33.739772 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:47:33.739775 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:47:33.739917 (+ 142us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:47:33.739993 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:47:33.739999 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:47:33.740768 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:47:33.740771 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:47:33.741549 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:47:33.741557 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:47:33.742051 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:47:33.742055 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:47:33.742122 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:33.743117 (+ 995us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:47:33.743129 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:47:33.744200 (+ 1071us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:47:33.746107 (+ 1907us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:47:33.746110 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:47:33.746112 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:47:33.746352 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:47:33.746643 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:47:33.746646 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:47:33.746648 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:47:33.746738 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:47:33.747188 (+ 450us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:47:33.747191 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:47:33.747258 (+ 67us) client_negotiation.cc:769] Sending connection context
0925 02:47:33.747336 (+ 78us) client_negotiation.cc:240] Negotiation successful
0925 02:47:33.747474 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260925 02:47:33.747875 4477 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.157.188" port: 36985 } attempt: 3
I20260925 02:47:33.748955 4482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:33.740066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58930 (local address 127.13.157.189:45923)
0925 02:47:33.740170 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:33.740174 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:33.740185 (+ 11us) server_negotiation.cc:407] Connection header received
0925 02:47:33.740240 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:33.740243 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:33.740282 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:33.740349 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:47:33.740811 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:33.741543 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:33.742510 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:33.742850 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:33.743005 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:33.745350 (+ 2345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:47:33.745354 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:47:33.745357 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:47:33.745381 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:47:33.747073 (+ 1692us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:47:33.747550 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:47:33.747553 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:47:33.747555 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:47:33.747635 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:47:33.748147 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:47:33.748151 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:47:33.748153 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:47:33.748374 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:47:33.748572 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:33.748705 (+ 133us) server_negotiation.cc:299] Negotiation successful
0925 02:47:33.748814 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
Sep 25 02:47:33 dist-test-slave-kznw krb5kdc[4321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304453, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.190@KRBTEST.COM for kudu/127.13.157.188@KRBTEST.COM
I20260925 02:47:33.752221 4553 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f0f0a05d42b43e88eb8ac891312b84d: Bootstrap starting.
I20260925 02:47:33.752853 13942 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 02:47:33.754527 4404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:33.745853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.188:36985 (local address 127.0.0.1:38598)
0925 02:47:33.746045 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0925 02:47:33.746057 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 02:47:33.746141 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:47:33.746405 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:47:33.746409 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:47:33.746581 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:47:33.746678 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:47:33.746684 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:47:33.748003 (+ 1319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:47:33.748006 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:47:33.748721 (+ 715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:47:33.748732 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:47:33.749232 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:47:33.749236 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:47:33.749316 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:33.749718 (+ 402us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:47:33.749731 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:47:33.751162 (+ 1431us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:47:33.752992 (+ 1830us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:47:33.752995 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:47:33.752997 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:47:33.753236 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:47:33.753521 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:47:33.753524 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:47:33.753526 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:47:33.753619 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:47:33.754016 (+ 397us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:47:33.754019 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:47:33.754122 (+ 103us) client_negotiation.cc:769] Sending connection context
0925 02:47:33.754227 (+ 105us) client_negotiation.cc:240] Negotiation successful
0925 02:47:33.754371 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111}
I20260925 02:47:33.754572 4558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:33.745672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38598 (local address 127.13.157.188:36985)
0925 02:47:33.746010 (+ 338us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:33.746015 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:33.746119 (+ 104us) server_negotiation.cc:407] Connection header received
0925 02:47:33.746235 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:33.746238 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:33.746283 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:33.746363 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:47:33.746769 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:33.747877 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:33.748857 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:33.749128 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:33.749211 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:33.751302 (+ 2091us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:47:33.751306 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:47:33.751308 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:47:33.751329 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:47:33.752881 (+ 1552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:47:33.753333 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:47:33.753337 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:47:33.753338 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:47:33.753432 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:47:33.753708 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:47:33.753711 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:47:33.753712 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:47:33.753937 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:47:33.754027 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:33.754284 (+ 257us) server_negotiation.cc:299] Negotiation successful
0925 02:47:33.754403 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":274,"thread_start_us":135,"threads_started":1}
I20260925 02:47:33.755129 4553 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1f0f0a05d42b43e88eb8ac891312b84d: Neither blocks nor log segments found. Creating new log.
I20260925 02:47:33.754067 4376 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "8c95418adbd04368bfda33dd9bd3d64d"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49796
I20260925 02:47:33.755885 4553 log.cc:824] T 00000000000000000000000000000000 P 1f0f0a05d42b43e88eb8ac891312b84d: Log is configured to *not* fsync() on all Append() calls
Sep 25 02:47:33 dist-test-slave-kznw krb5kdc[4321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790304453, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.157.189@KRBTEST.COM for kudu/127.13.157.188@KRBTEST.COM
I20260925 02:47:33.757614 4402 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c95418adbd04368bfda33dd9bd3d64d: Bootstrap starting.
I20260925 02:47:33.758340 4553 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f0f0a05d42b43e88eb8ac891312b84d: No bootstrap required, opened a new log
I20260925 02:47:33.760460 4402 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8c95418adbd04368bfda33dd9bd3d64d: Neither blocks nor log segments found. Creating new log.
I20260925 02:47:33.761267 4480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:33.748295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.157.188:36985 (local address 127.0.0.1:38600)
0925 02:47:33.748457 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0925 02:47:33.748469 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 02:47:33.748529 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:47:33.750141 (+ 1612us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:47:33.750144 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:47:33.750310 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:47:33.750405 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:47:33.750412 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:47:33.752037 (+ 1625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:47:33.752040 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:47:33.752757 (+ 717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:47:33.752766 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:47:33.755850 (+ 3084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:47:33.755854 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:47:33.755947 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:33.756407 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:47:33.756423 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:47:33.757768 (+ 1345us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:47:33.759646 (+ 1878us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:47:33.759667 (+ 21us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:47:33.759670 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:47:33.759925 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:47:33.760198 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:47:33.760201 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:47:33.760203 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:47:33.760294 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:47:33.760711 (+ 417us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:47:33.760714 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:47:33.760850 (+ 136us) client_negotiation.cc:769] Sending connection context
0925 02:47:33.760962 (+ 112us) client_negotiation.cc:240] Negotiation successful
0925 02:47:33.761099 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101}
I20260925 02:47:33.761272 4557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:33.748359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38600 (local address 127.13.157.188:36985)
0925 02:47:33.749636 (+ 1277us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:33.749640 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:33.749652 (+ 12us) server_negotiation.cc:407] Connection header received
0925 02:47:33.749681 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:33.749684 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:33.749721 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:33.750835 (+ 1114us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:47:33.751124 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:33.751751 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:33.753874 (+ 2123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:33.755732 (+ 1858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:33.755835 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:33.757910 (+ 2075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:47:33.757913 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:47:33.757916 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:47:33.757940 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:47:33.759537 (+ 1597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:47:33.760016 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:47:33.760019 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:47:33.760020 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:47:33.760103 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:47:33.760387 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:47:33.760391 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:47:33.760392 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:47:33.760603 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:47:33.760698 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:33.761022 (+ 324us) server_negotiation.cc:299] Negotiation successful
0925 02:47:33.761133 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1211}
I20260925 02:47:33.761315 4402 log.cc:824] T 00000000000000000000000000000000 P 8c95418adbd04368bfda33dd9bd3d64d: Log is configured to *not* fsync() on all Append() calls
I20260925 02:47:33.762238 4553 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1f0f0a05d42b43e88eb8ac891312b84d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c95418adbd04368bfda33dd9bd3d64d" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 43961 } } peers { permanent_uuid: "f120b693139f461d98983032354be924" member_type: VOTER last_known_addr { host: "127.13.157.189" port: 45923 } } peers { permanent_uuid: "1f0f0a05d42b43e88eb8ac891312b84d" member_type: VOTER last_known_addr { host: "127.13.157.188" port: 36985 } }
I20260925 02:47:33.762496 4553 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1f0f0a05d42b43e88eb8ac891312b84d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f0f0a05d42b43e88eb8ac891312b84d, State: Initialized, Role: FOLLOWER
I20260925 02:47:33.762801 4553 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1f0f0a05d42b43e88eb8ac891312b84d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c95418adbd04368bfda33dd9bd3d64d" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 43961 } } peers { permanent_uuid: "f120b693139f461d98983032354be924" member_type: VOTER last_known_addr { host: "127.13.157.189" port: 45923 } } peers { permanent_uuid: "1f0f0a05d42b43e88eb8ac891312b84d" member_type: VOTER last_known_addr { host: "127.13.157.188" port: 36985 } }
I20260925 02:47:33.763279 4561 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f0f0a05d42b43e88eb8ac891312b84d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c95418adbd04368bfda33dd9bd3d64d" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 43961 } } peers { permanent_uuid: "f120b693139f461d98983032354be924" member_type: VOTER last_known_addr { host: "127.13.157.189" port: 45923 } } peers { permanent_uuid: "1f0f0a05d42b43e88eb8ac891312b84d" member_type: VOTER last_known_addr { host: "127.13.157.188" port: 36985 } } }
I20260925 02:47:33.763403 4561 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f0f0a05d42b43e88eb8ac891312b84d [sys.catalog]: This master's current role is: FOLLOWER
I20260925 02:47:33.764058 4553 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1f0f0a05d42b43e88eb8ac891312b84d [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:47:33.764204 4402 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c95418adbd04368bfda33dd9bd3d64d: No bootstrap required, opened a new log
I20260925 02:47:33.764424 4477 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f120b693139f461d98983032354be924: Bootstrap starting.
I20260925 02:47:33.767232 4402 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8c95418adbd04368bfda33dd9bd3d64d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c95418adbd04368bfda33dd9bd3d64d" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 43961 } } peers { permanent_uuid: "f120b693139f461d98983032354be924" member_type: VOTER last_known_addr { host: "127.13.157.189" port: 45923 } } peers { permanent_uuid: "1f0f0a05d42b43e88eb8ac891312b84d" member_type: VOTER last_known_addr { host: "127.13.157.188" port: 36985 } }
I20260925 02:47:33.767508 4402 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8c95418adbd04368bfda33dd9bd3d64d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c95418adbd04368bfda33dd9bd3d64d, State: Initialized, Role: FOLLOWER
I20260925 02:47:33.767874 4402 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8c95418adbd04368bfda33dd9bd3d64d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c95418adbd04368bfda33dd9bd3d64d" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 43961 } } peers { permanent_uuid: "f120b693139f461d98983032354be924" member_type: VOTER last_known_addr { host: "127.13.157.189" port: 45923 } } peers { permanent_uuid: "1f0f0a05d42b43e88eb8ac891312b84d" member_type: VOTER last_known_addr { host: "127.13.157.188" port: 36985 } }
W20260925 02:47:33.768457 4572 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1f0f0a05d42b43e88eb8ac891312b84d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 02:47:33.768533 4572 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 02:47:33.768765 4574 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c95418adbd04368bfda33dd9bd3d64d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c95418adbd04368bfda33dd9bd3d64d" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 43961 } } peers { permanent_uuid: "f120b693139f461d98983032354be924" member_type: VOTER last_known_addr { host: "127.13.157.189" port: 45923 } } peers { permanent_uuid: "1f0f0a05d42b43e88eb8ac891312b84d" member_type: VOTER last_known_addr { host: "127.13.157.188" port: 36985 } } }
I20260925 02:47:33.768910 4574 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c95418adbd04368bfda33dd9bd3d64d [sys.catalog]: This master's current role is: FOLLOWER
I20260925 02:47:33.769472 4402 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8c95418adbd04368bfda33dd9bd3d64d [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:47:33.767464 4484 master_runner.cc:186] Error getting master registration for 127.13.157.190:43961: OK, instance_id { permanent_uuid: "8c95418adbd04368bfda33dd9bd3d64d" instance_seqno: 1790304453279696 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260925 02:47:33.770619 4484 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260925 02:47:33.770776 4376 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8c95418adbd04368bfda33dd9bd3d64d [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260925 02:47:33.770885 4376 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8c95418adbd04368bfda33dd9bd3d64d [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:47:33.772094 4376 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8c95418adbd04368bfda33dd9bd3d64d [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c95418adbd04368bfda33dd9bd3d64d" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 43961 } } peers { permanent_uuid: "f120b693139f461d98983032354be924" member_type: VOTER last_known_addr { host: "127.13.157.189" port: 45923 } } peers { permanent_uuid: "1f0f0a05d42b43e88eb8ac891312b84d" member_type: VOTER last_known_addr { host: "127.13.157.188" port: 36985 } }
I20260925 02:47:33.774232 4477 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f120b693139f461d98983032354be924: Neither blocks nor log segments found. Creating new log.
I20260925 02:47:33.775202 4477 log.cc:824] T 00000000000000000000000000000000 P f120b693139f461d98983032354be924: Log is configured to *not* fsync() on all Append() calls
W20260925 02:47:33.775799 4585 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8c95418adbd04368bfda33dd9bd3d64d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 02:47:33.775871 4585 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 02:47:33.775974 4376 leader_election.cc:288] T 00000000000000000000000000000000 P 8c95418adbd04368bfda33dd9bd3d64d [CANDIDATE]: Term 1 election: Requested vote from peers f120b693139f461d98983032354be924 (127.13.157.189:45923), 1f0f0a05d42b43e88eb8ac891312b84d (127.13.157.188:36985)
W20260925 02:47:33.776343 4355 tablet.cc:2369] T 00000000000000000000000000000000 P 8c95418adbd04368bfda33dd9bd3d64d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 02:47:33.777711 4477 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f120b693139f461d98983032354be924: No bootstrap required, opened a new log
I20260925 02:47:33.775398 4334 master_runner.cc:186] Error getting master registration for 127.13.157.189:45923: OK, instance_id { permanent_uuid: "f120b693139f461d98983032354be924" instance_seqno: 1790304453495336 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260925 02:47:33.778038 4334 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260925 02:47:33.779425 4527 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8c95418adbd04368bfda33dd9bd3d64d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "1f0f0a05d42b43e88eb8ac891312b84d"
I20260925 02:47:33.779707 4527 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1f0f0a05d42b43e88eb8ac891312b84d [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:47:33.780894 4527 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1f0f0a05d42b43e88eb8ac891312b84d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c95418adbd04368bfda33dd9bd3d64d in term 1.
I20260925 02:47:33.781328 4351 leader_election.cc:302] T 00000000000000000000000000000000 P 8c95418adbd04368bfda33dd9bd3d64d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f0f0a05d42b43e88eb8ac891312b84d, 8c95418adbd04368bfda33dd9bd3d64d; no voters:
I20260925 02:47:33.781633 4574 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8c95418adbd04368bfda33dd9bd3d64d [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:47:33.781700 4477 raft_consensus.cc:348] T 00000000000000000000000000000000 P f120b693139f461d98983032354be924 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c95418adbd04368bfda33dd9bd3d64d" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 43961 } } peers { permanent_uuid: "f120b693139f461d98983032354be924" member_type: VOTER last_known_addr { host: "127.13.157.189" port: 45923 } } peers { permanent_uuid: "1f0f0a05d42b43e88eb8ac891312b84d" member_type: VOTER last_known_addr { host: "127.13.157.188" port: 36985 } }
I20260925 02:47:33.781827 4574 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8c95418adbd04368bfda33dd9bd3d64d [term 1 LEADER]: Becoming Leader. State: Replica: 8c95418adbd04368bfda33dd9bd3d64d, State: Running, Role: LEADER
I20260925 02:47:33.781905 4477 raft_consensus.cc:729] T 00000000000000000000000000000000 P f120b693139f461d98983032354be924 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f120b693139f461d98983032354be924, State: Initialized, Role: FOLLOWER
I20260925 02:47:33.782158 4574 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8c95418adbd04368bfda33dd9bd3d64d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c95418adbd04368bfda33dd9bd3d64d" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 43961 } } peers { permanent_uuid: "f120b693139f461d98983032354be924" member_type: VOTER last_known_addr { host: "127.13.157.189" port: 45923 } } peers { permanent_uuid: "1f0f0a05d42b43e88eb8ac891312b84d" member_type: VOTER last_known_addr { host: "127.13.157.188" port: 36985 } }
I20260925 02:47:33.782209 4477 consensus_queue.cc:260] T 00000000000000000000000000000000 P f120b693139f461d98983032354be924 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c95418adbd04368bfda33dd9bd3d64d" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 43961 } } peers { permanent_uuid: "f120b693139f461d98983032354be924" member_type: VOTER last_known_addr { host: "127.13.157.189" port: 45923 } } peers { permanent_uuid: "1f0f0a05d42b43e88eb8ac891312b84d" member_type: VOTER last_known_addr { host: "127.13.157.188" port: 36985 } }
I20260925 02:47:33.783524 4477 sys_catalog.cc:564] T 00000000000000000000000000000000 P f120b693139f461d98983032354be924 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:47:33.783910 4450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8c95418adbd04368bfda33dd9bd3d64d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "f120b693139f461d98983032354be924"
I20260925 02:47:33.783952 4574 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c95418adbd04368bfda33dd9bd3d64d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8c95418adbd04368bfda33dd9bd3d64d. Latest consensus state: current_term: 1 leader_uuid: "8c95418adbd04368bfda33dd9bd3d64d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c95418adbd04368bfda33dd9bd3d64d" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 43961 } } peers { permanent_uuid: "f120b693139f461d98983032354be924" member_type: VOTER last_known_addr { host: "127.13.157.189" port: 45923 } } peers { permanent_uuid: "1f0f0a05d42b43e88eb8ac891312b84d" member_type: VOTER last_known_addr { host: "127.13.157.188" port: 36985 } } }
I20260925 02:47:33.784055 4574 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c95418adbd04368bfda33dd9bd3d64d [sys.catalog]: This master's current role is: LEADER
I20260925 02:47:33.784124 4450 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f120b693139f461d98983032354be924 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:47:33.784649 4597 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:47:33.785406 4450 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f120b693139f461d98983032354be924 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c95418adbd04368bfda33dd9bd3d64d in term 1.
I20260925 02:47:33.787375 4597 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:47:33.791164 4593 sys_catalog.cc:455] T 00000000000000000000000000000000 P f120b693139f461d98983032354be924 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c95418adbd04368bfda33dd9bd3d64d" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 43961 } } peers { permanent_uuid: "f120b693139f461d98983032354be924" member_type: VOTER last_known_addr { host: "127.13.157.189" port: 45923 } } peers { permanent_uuid: "1f0f0a05d42b43e88eb8ac891312b84d" member_type: VOTER last_known_addr { host: "127.13.157.188" port: 36985 } } }
I20260925 02:47:33.791312 4593 sys_catalog.cc:458] T 00000000000000000000000000000000 P f120b693139f461d98983032354be924 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 02:47:33.797907 4405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:33.781920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49810 (local address 127.13.157.190:43961)
0925 02:47:33.782029 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:33.782033 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:33.787771 (+ 5738us) server_negotiation.cc:407] Connection header received
0925 02:47:33.787945 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:33.787948 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:33.787995 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:33.788040 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:47:33.790465 (+ 2425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:33.791156 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:33.792228 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:33.792555 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:33.792631 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:33.794225 (+ 1594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:47:33.794229 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:47:33.794231 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:47:33.794257 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:47:33.795824 (+ 1567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:47:33.796307 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:47:33.796310 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:47:33.796312 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:47:33.796411 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:47:33.796925 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:47:33.796929 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:47:33.796930 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:47:33.797156 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:47:33.797265 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:33.797631 (+ 366us) server_negotiation.cc:299] Negotiation successful
0925 02:47:33.797749 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260925 02:47:33.801985 4482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:33.782260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58940 (local address 127.13.157.189:45923)
0925 02:47:33.782372 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:33.782376 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:33.786192 (+ 3816us) server_negotiation.cc:407] Connection header received
0925 02:47:33.786404 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:33.786408 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:33.786463 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:33.786517 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:47:33.788946 (+ 2429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:33.789623 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:33.790441 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:33.790755 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:33.790848 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:33.797074 (+ 6226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:47:33.797077 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:47:33.797080 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:47:33.797104 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:47:33.800379 (+ 3275us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:47:33.800838 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:47:33.800841 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:47:33.800843 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:47:33.800936 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:47:33.801193 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:47:33.801196 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:47:33.801198 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:47:33.801378 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:47:33.801485 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:33.801718 (+ 233us) server_negotiation.cc:299] Negotiation successful
0925 02:47:33.801829 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260925 02:47:33.807374 4557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:33.783489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38604 (local address 127.13.157.188:36985)
0925 02:47:33.783601 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:33.783605 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:33.791646 (+ 8041us) server_negotiation.cc:407] Connection header received
0925 02:47:33.791839 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:33.791843 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:33.791894 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:33.791946 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:47:33.793001 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:33.793747 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:33.794940 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:33.795273 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:33.795348 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:33.799867 (+ 4519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:47:33.799871 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:47:33.799874 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:47:33.799902 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:47:33.801436 (+ 1534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:47:33.801930 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:47:33.801933 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:47:33.801935 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:47:33.802019 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:47:33.802298 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:47:33.802302 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:47:33.802304 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:47:33.802511 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:47:33.802600 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:33.807109 (+ 4509us) server_negotiation.cc:299] Negotiation successful
0925 02:47:33.807222 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
W20260925 02:47:33.810804 4610 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f120b693139f461d98983032354be924: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 02:47:33.810983 4610 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 02:47:33.812836 4527 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1f0f0a05d42b43e88eb8ac891312b84d [term 1 FOLLOWER]: Refusing update from remote peer 8c95418adbd04368bfda33dd9bd3d64d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:47:33.815196 4592 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8c95418adbd04368bfda33dd9bd3d64d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f0f0a05d42b43e88eb8ac891312b84d" member_type: VOTER last_known_addr { host: "127.13.157.188" port: 36985 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:47:33.819721 4561 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f0f0a05d42b43e88eb8ac891312b84d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8c95418adbd04368bfda33dd9bd3d64d. Latest consensus state: current_term: 1 leader_uuid: "8c95418adbd04368bfda33dd9bd3d64d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c95418adbd04368bfda33dd9bd3d64d" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 43961 } } peers { permanent_uuid: "f120b693139f461d98983032354be924" member_type: VOTER last_known_addr { host: "127.13.157.189" port: 45923 } } peers { permanent_uuid: "1f0f0a05d42b43e88eb8ac891312b84d" member_type: VOTER last_known_addr { host: "127.13.157.188" port: 36985 } } }
I20260925 02:47:33.819849 4561 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f0f0a05d42b43e88eb8ac891312b84d [sys.catalog]: This master's current role is: FOLLOWER
I20260925 02:47:33.820186 4450 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f120b693139f461d98983032354be924 [term 1 FOLLOWER]: Refusing update from remote peer 8c95418adbd04368bfda33dd9bd3d64d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:47:33.823112 4592 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8c95418adbd04368bfda33dd9bd3d64d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f120b693139f461d98983032354be924" member_type: VOTER last_known_addr { host: "127.13.157.189" port: 45923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:47:33.827330 4593 sys_catalog.cc:455] T 00000000000000000000000000000000 P f120b693139f461d98983032354be924 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8c95418adbd04368bfda33dd9bd3d64d. Latest consensus state: current_term: 1 leader_uuid: "8c95418adbd04368bfda33dd9bd3d64d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c95418adbd04368bfda33dd9bd3d64d" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 43961 } } peers { permanent_uuid: "f120b693139f461d98983032354be924" member_type: VOTER last_known_addr { host: "127.13.157.189" port: 45923 } } peers { permanent_uuid: "1f0f0a05d42b43e88eb8ac891312b84d" member_type: VOTER last_known_addr { host: "127.13.157.188" port: 36985 } } }
I20260925 02:47:33.827479 4593 sys_catalog.cc:458] T 00000000000000000000000000000000 P f120b693139f461d98983032354be924 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 02:47:33.830359 4574 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c95418adbd04368bfda33dd9bd3d64d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8c95418adbd04368bfda33dd9bd3d64d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c95418adbd04368bfda33dd9bd3d64d" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 43961 } } peers { permanent_uuid: "f120b693139f461d98983032354be924" member_type: VOTER last_known_addr { host: "127.13.157.189" port: 45923 } } peers { permanent_uuid: "1f0f0a05d42b43e88eb8ac891312b84d" member_type: VOTER last_known_addr { host: "127.13.157.188" port: 36985 } } }
I20260925 02:47:33.830493 4574 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c95418adbd04368bfda33dd9bd3d64d [sys.catalog]: This master's current role is: LEADER
I20260925 02:47:33.830694 4612 mvcc.cc:205] Tried to move back new op lower bound from 7333087042806083584 to 7333087042695311360. Current Snapshot: MvccSnapshot[applied={T|T < 7333087042806083584}]
I20260925 02:47:33.830864 4561 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f0f0a05d42b43e88eb8ac891312b84d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8c95418adbd04368bfda33dd9bd3d64d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c95418adbd04368bfda33dd9bd3d64d" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 43961 } } peers { permanent_uuid: "f120b693139f461d98983032354be924" member_type: VOTER last_known_addr { host: "127.13.157.189" port: 45923 } } peers { permanent_uuid: "1f0f0a05d42b43e88eb8ac891312b84d" member_type: VOTER last_known_addr { host: "127.13.157.188" port: 36985 } } }
I20260925 02:47:33.831004 4561 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f0f0a05d42b43e88eb8ac891312b84d [sys.catalog]: This master's current role is: FOLLOWER
I20260925 02:47:33.830955 4609 mvcc.cc:205] Tried to move back new op lower bound from 7333087042806083584 to 7333087042695311360. Current Snapshot: MvccSnapshot[applied={T|T < 7333087042806083584}]
I20260925 02:47:33.839131 4592 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c95418adbd04368bfda33dd9bd3d64d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8c95418adbd04368bfda33dd9bd3d64d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c95418adbd04368bfda33dd9bd3d64d" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 43961 } } peers { permanent_uuid: "f120b693139f461d98983032354be924" member_type: VOTER last_known_addr { host: "127.13.157.189" port: 45923 } } peers { permanent_uuid: "1f0f0a05d42b43e88eb8ac891312b84d" member_type: VOTER last_known_addr { host: "127.13.157.188" port: 36985 } } }
I20260925 02:47:33.839279 4592 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c95418adbd04368bfda33dd9bd3d64d [sys.catalog]: This master's current role is: LEADER
I20260925 02:47:33.842747 4593 sys_catalog.cc:455] T 00000000000000000000000000000000 P f120b693139f461d98983032354be924 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8c95418adbd04368bfda33dd9bd3d64d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c95418adbd04368bfda33dd9bd3d64d" member_type: VOTER last_known_addr { host: "127.13.157.190" port: 43961 } } peers { permanent_uuid: "f120b693139f461d98983032354be924" member_type: VOTER last_known_addr { host: "127.13.157.189" port: 45923 } } peers { permanent_uuid: "1f0f0a05d42b43e88eb8ac891312b84d" member_type: VOTER last_known_addr { host: "127.13.157.188" port: 36985 } } }
I20260925 02:47:33.842875 4593 sys_catalog.cc:458] T 00000000000000000000000000000000 P f120b693139f461d98983032354be924 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 02:47:33.845546 4614 mvcc.cc:205] Tried to move back new op lower bound from 7333087042806083584 to 7333087042695311360. Current Snapshot: MvccSnapshot[applied={T|T < 7333087042806083584}]
I20260925 02:47:33.846107 4597 catalog_manager.cc:1347] Generated new cluster ID: b53a9bef1c9d4f228f9f04bb7f353687
I20260925 02:47:33.846170 4597 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 02:47:33.899483 4597 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 02:47:33.900785 4597 catalog_manager.cc:1504] Loading token signing keys...
I20260925 02:47:33.917018 4597 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8c95418adbd04368bfda33dd9bd3d64d: Generated new TSK 0
I20260925 02:47:33.917816 4597 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260925 02:47:33.954576 4482 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:47:33.951603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58942 (local address 127.13.157.189:45923)
0925 02:47:33.951724 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:33.951729 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:33.953438 (+ 1709us) server_negotiation.cc:407] Connection header received
0925 02:47:33.953705 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:33.953708 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:33.953761 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:33.953812 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:47:33.954436 (+ 624us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58942: BlockingRecv error: recv got EOF from 127.0.0.1:58942 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260925 02:47:33.955952 4557 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:47:33.952267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38618 (local address 127.13.157.188:36985)
0925 02:47:33.952375 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:33.952379 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:33.955086 (+ 2707us) server_negotiation.cc:407] Connection header received
0925 02:47:33.955249 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:33.955252 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:33.955301 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:33.955350 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:47:33.955833 (+ 483us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38618: BlockingRecv error: recv got EOF from 127.0.0.1:38618 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
W20260925 02:47:33.957257 4405 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:47:33.952825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49820 (local address 127.13.157.190:43961)
0925 02:47:33.952943 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:33.952947 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:33.956374 (+ 3427us) server_negotiation.cc:407] Connection header received
0925 02:47:33.956565 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:33.956569 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:33.956620 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:33.956669 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:47:33.957139 (+ 470us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49820: BlockingRecv error: recv got EOF from 127.0.0.1:49820 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260925 02:47:33.961378 4482 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:47:33.959677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58958 (local address 127.13.157.189:45923)
0925 02:47:33.959796 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:33.959800 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:33.960433 (+ 633us) server_negotiation.cc:407] Connection header received
0925 02:47:33.960660 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:33.960663 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:33.960739 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:33.960788 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:47:33.961279 (+ 491us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58958: BlockingRecv error: recv got EOF from 127.0.0.1:58958 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
W20260925 02:47:33.966513 4482 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:47:33.964954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58970 (local address 127.13.157.189:45923)
0925 02:47:33.965068 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:33.965072 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:33.965668 (+ 596us) server_negotiation.cc:407] Connection header received
0925 02:47:33.965853 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:33.965856 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:33.965904 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:33.965949 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:47:33.966406 (+ 457us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58970: BlockingRecv error: recv got EOF from 127.0.0.1:58970 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260925 02:47:33.972417 4482 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:47:33.970650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58986 (local address 127.13.157.189:45923)
0925 02:47:33.970759 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:33.970763 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:33.971391 (+ 628us) server_negotiation.cc:407] Connection header received
0925 02:47:33.971643 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:33.971646 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:33.971697 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:33.971771 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:47:33.972319 (+ 548us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58986: BlockingRecv error: recv got EOF from 127.0.0.1:58986 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
W20260925 02:47:33.980484 4482 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:47:33.978661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59002 (local address 127.13.157.189:45923)
0925 02:47:33.978797 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:33.978800 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:33.979407 (+ 607us) server_negotiation.cc:407] Connection header received
0925 02:47:33.979638 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:33.979642 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:33.979697 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:33.979750 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:47:33.980361 (+ 611us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59002: BlockingRecv error: recv got EOF from 127.0.0.1:59002 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
W20260925 02:47:33.991323 4482 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:47:33.990242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59018 (local address 127.13.157.189:45923)
0925 02:47:33.990356 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:33.990360 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:33.990506 (+ 146us) server_negotiation.cc:407] Connection header received
0925 02:47:33.990628 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:33.990630 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:33.990676 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:33.990743 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:47:33.991210 (+ 467us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59018: BlockingRecv error: recv got EOF from 127.0.0.1:59018 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260925 02:47:34.011134 4482 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:47:34.009607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59030 (local address 127.13.157.189:45923)
0925 02:47:34.009755 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:34.009760 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:34.010116 (+ 356us) server_negotiation.cc:407] Connection header received
0925 02:47:34.010374 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:34.010378 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:34.010441 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:34.010531 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:47:34.011022 (+ 491us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59030: BlockingRecv error: recv got EOF from 127.0.0.1:59030 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
W20260925 02:47:34.046608 4482 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:47:34.045495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59040 (local address 127.13.157.189:45923)
0925 02:47:34.045629 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:34.045633 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:34.045727 (+ 94us) server_negotiation.cc:407] Connection header received
0925 02:47:34.045857 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:34.045860 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:34.045929 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:34.046035 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:47:34.046509 (+ 474us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59040: BlockingRecv error: recv got EOF from 127.0.0.1:59040 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
W20260925 02:47:34.114130 4482 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:47:34.112786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59052 (local address 127.13.157.189:45923)
0925 02:47:34.112961 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:34.112965 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:34.113221 (+ 256us) server_negotiation.cc:407] Connection header received
0925 02:47:34.113348 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:34.113351 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:34.113410 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:34.113492 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:47:34.114008 (+ 516us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59052: BlockingRecv error: recv got EOF from 127.0.0.1:59052 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
W20260925 02:47:34.245843 4482 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:47:34.244516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59064 (local address 127.13.157.189:45923)
0925 02:47:34.244687 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:34.244691 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:34.244855 (+ 164us) server_negotiation.cc:407] Connection header received
0925 02:47:34.245104 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:34.245107 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:34.245170 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:34.245253 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:47:34.245740 (+ 487us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59064: BlockingRecv error: recv got EOF from 127.0.0.1:59064 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
W20260925 02:47:34.505205 4482 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:47:34.504046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59080 (local address 127.13.157.189:45923)
0925 02:47:34.504176 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:34.504180 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:34.504360 (+ 180us) server_negotiation.cc:407] Connection header received
0925 02:47:34.504503 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:34.504507 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:34.504558 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:34.504622 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:47:34.505095 (+ 473us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59080: BlockingRecv error: recv got EOF from 127.0.0.1:59080 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
I20260925 02:47:34.769954 4572 catalog_manager.cc:1259] Loaded cluster ID: b53a9bef1c9d4f228f9f04bb7f353687
I20260925 02:47:34.770040 4572 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1f0f0a05d42b43e88eb8ac891312b84d: loading cluster ID for follower catalog manager: success
I20260925 02:47:34.772047 4572 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1f0f0a05d42b43e88eb8ac891312b84d: acquiring CA information for follower catalog manager: success
I20260925 02:47:34.773200 4572 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1f0f0a05d42b43e88eb8ac891312b84d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 02:47:34.812398 4610 catalog_manager.cc:1259] Loaded cluster ID: b53a9bef1c9d4f228f9f04bb7f353687
I20260925 02:47:34.812494 4610 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f120b693139f461d98983032354be924: loading cluster ID for follower catalog manager: success
I20260925 02:47:34.814201 4610 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f120b693139f461d98983032354be924: acquiring CA information for follower catalog manager: success
I20260925 02:47:34.815392 4610 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f120b693139f461d98983032354be924: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 02:47:35.023541 4683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:35.019541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59092 (local address 127.13.157.189:45923)
0925 02:47:35.019791 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:35.019795 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:35.019894 (+ 99us) server_negotiation.cc:407] Connection header received
0925 02:47:35.020013 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:35.020015 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:35.020142 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:35.020240 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 02:47:35.020527 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:35.021081 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:35.021955 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:35.022173 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:35.022244 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:35.022582 (+ 338us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 02:47:35.023107 (+ 525us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 02:47:35.023216 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:35.023297 (+ 81us) server_negotiation.cc:299] Negotiation successful
0925 02:47:35.023362 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":80,"threads_started":1}
I20260925 02:47:35.030648 4689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:35.026834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38632 (local address 127.13.157.188:36985)
0925 02:47:35.027099 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:35.027103 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:35.027116 (+ 13us) server_negotiation.cc:407] Connection header received
0925 02:47:35.027195 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:35.027198 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:35.027312 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:35.027448 (+ 136us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 02:47:35.027679 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:35.028191 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:35.029078 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:35.029309 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:35.029446 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:35.029716 (+ 270us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 02:47:35.030170 (+ 454us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 02:47:35.030302 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:35.030431 (+ 129us) server_negotiation.cc:299] Negotiation successful
0925 02:47:35.030484 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":89,"threads_started":1}
I20260925 02:47:35.037523 4683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:35.033878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59094 (local address 127.13.157.189:45923)
0925 02:47:35.034035 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:35.034040 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:35.034156 (+ 116us) server_negotiation.cc:407] Connection header received
0925 02:47:35.034311 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:35.034314 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:35.034366 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:35.034449 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 02:47:35.034649 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:35.035305 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:35.036426 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:35.036746 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:35.036841 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:35.037075 (+ 234us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 02:47:35.037151 (+ 76us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 02:47:35.037190 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:35.037348 (+ 158us) server_negotiation.cc:299] Negotiation successful
0925 02:47:35.037403 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260925 02:47:35.038044 4689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:35.034238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38636 (local address 127.13.157.188:36985)
0925 02:47:35.034367 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:35.034371 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:35.034799 (+ 428us) server_negotiation.cc:407] Connection header received
0925 02:47:35.034934 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:35.034938 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:35.034989 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:35.035056 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 02:47:35.035313 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:35.035814 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:35.036946 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:35.037242 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:35.037337 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:35.037568 (+ 231us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 02:47:35.037639 (+ 71us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 02:47:35.037704 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:35.037841 (+ 137us) server_negotiation.cc:299] Negotiation successful
0925 02:47:35.037927 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260925 02:47:35.038719 4695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:47:35.033697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49826 (local address 127.13.157.190:43961)
0925 02:47:35.034092 (+ 395us) server_negotiation.cc:206] Beginning negotiation
0925 02:47:35.034098 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 02:47:35.034115 (+ 17us) server_negotiation.cc:407] Connection header received
0925 02:47:35.034224 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:47:35.034228 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:47:35.034398 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:47:35.034530 (+ 132us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 02:47:35.034969 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:35.035754 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:35.036776 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:47:35.037075 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:47:35.037165 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:47:35.037512 (+ 347us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 02:47:35.038006 (+ 494us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 02:47:35.038091 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0925 02:47:35.038218 (+ 127us) server_negotiation.cc:299] Negotiation successful
0925 02:47:35.038273 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":60,"threads_started":1}
I20260925 02:47:35.041482 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 4334
I20260925 02:47:35.047642 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 4409
I20260925 02:47:35.053074 13942 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksiQ9fF/build/debug/bin/kudu with pid 4484
2026-09-25T02:47:35Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3028 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6069 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (276572 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST